Brain function intelligent monitoring method and system based on multi-modal data

Abstract

The invention relates to the technical field of brain data monitoring, and discloses a brain function intelligent monitoring method and system based on multi-modal data, and the method comprises the steps: collecting brain multi-modal data of a user through a head wearing device; determining a brain dynamic graph object of the user according to the brain multi-modal data, and performing feature fusion operation on the brain dynamic graph object according to the brain dynamic graph object to obtain a brain multi-modal feature fusion result of the user; and determining brain function monitoring data of the user according to a brain multi-modal feature fusion result. It can be seen that the brain function monitoring data of the user can be determined by collecting and intelligently analyzing the brain multi-modal data of the user through the portable head wearable device, so that dependence on large-scale brain monitoring equipment is reduced, and the user experience is improved. And the dynamic change condition of the brain function of the user is accurately reflected through the multi-modal data, so that the monitoring accuracy of the brain function data of the user is improved.

Description

Technical Field

[0001] This invention relates to the field of brain data monitoring technology, and in particular to a method and system for intelligent monitoring of brain function based on multimodal data. Background Technology

[0002] In the field of brain data monitoring research, traditional techniques primarily revolve around large medical equipment such as MRI and PET. However, these devices are not only expensive to purchase but also incur consistently high maintenance costs. Furthermore, their complex operation requires highly skilled professionals, severely limiting their application scenarios and making them difficult to deploy in everyday environments such as homes or communities. This significantly restricts the acquisition of brain data monitoring data across a wider range of populations and scenarios, thus slowing down research progress. In addition, because the brain data collected from the monitored subjects is relatively singular, it is difficult to comprehensively and accurately reflect the complex state and dynamic changes of brain function. This makes the analysis and research based on this single data set highly limited, further constraining the efficiency and quality of brain data monitoring research. Therefore, providing a method to improve the accuracy of brain function data monitoring is of paramount importance. Summary of the Invention

[0003] This invention provides a method and system for intelligent monitoring of brain function based on multimodal data. It not only reduces the reliance on large-scale brain monitoring equipment, but also accurately reflects the dynamic changes in the user's brain function through multimodal data, thereby improving the reliability and accuracy of monitoring the user's brain function data.

[0004] To address the aforementioned technical problems, the first aspect of this invention discloses a method for intelligent monitoring of brain function based on multimodal data, the method comprising: The system collects multimodal brain data from users through a pre-set head-worn device. The head-worn device integrates an MRI scanner, an EEG parameter acquisition device, and a hemoglobin parameter acquisition device. The multimodal brain data includes gray matter volume data, EEG data, and hemoglobin concentration data. Based on the brain multimodal data, the user's brain motion map object is determined, and based on the brain motion map object, a feature fusion operation is performed on the brain motion map object to obtain the user's brain multimodal feature fusion result; Based on the results of the fusion of the brain's multimodal features, the user's brain function monitoring data is determined.

[0005] As an optional implementation, in the first aspect of the invention, before collecting the user's multimodal brain data through a preset head-worn device, the method further includes: Based on the preset gradient magnetic field parameters of the MRI scanner in the head-worn device, determine the pulse test parameters for the EEG parameter acquisition device in the head-worn device. Based on the pulse test parameters, the interference parameters of the electrode pairs of the EEG parameter acquisition device are measured, and the pulse adjustment parameters of the EEG parameter acquisition device are determined based on the interference parameters of the electrode pairs. Based on the pulse adjustment parameters, the initial pulse parameters of the EEG parameter acquisition device are adjusted to obtain the adjusted pulse parameters of the EEG parameter acquisition device; The process of collecting multimodal brain data from a user via a pre-set head-worn device includes: Multimodal brain data of the user is collected through a pre-set head-worn device, based on the adjusted pulse parameters and the pre-set first acquisition control parameters of the MRI scanner, the second acquisition control parameters of the EEG parameter collector, and the third acquisition control parameters of the hemoglobin parameter collector.

[0006] As an optional implementation, in the first aspect of the present invention, the method further includes: During the process of collecting the user's EEG data, the first target parameters of the EEG parameter acquisition device and the second target parameters of the magnetic resonance imaging (MRI) device are obtained. The first target parameters of the EEG parameter acquisition device include the shielding coefficient of the EEG parameter acquisition device, the parameters of the effective EEG signals collected, and the parameters of the basic noise interference. The second target parameters of the MRI device include the magnetic field strength parameters and the gradient switching interval parameters of the MRI device. Based on the first target parameter and the second target parameter, the EEG signal-to-noise ratio parameter of the EEG parameter acquisition device is determined, and the EEG data is filtered based on the EEG signal-to-noise ratio parameter to update the EEG data; The signal-to-noise ratio parameter of the EEG signal is: SNR_EEG = 20*log10(V_signal / (V_noise+k*B_MRI*Δt); V_signal is the effective EEG signal parameter, V_noise is the basic noise interference parameter, k is the shielding coefficient, B_MRI is the magnetic field strength parameter, and Δt is the gradient switching interval parameter.

[0007] As an optional implementation, in a first aspect of the present invention, determining the user's brain motion map object based on the brain multimodal data includes: According to preset processing parameters, target processing operations are performed on the brain multimodal data to obtain multiple data sequences corresponding to the gray matter volume data, multiple first sliding window data groups corresponding to the EEG data, and multiple second sliding window data groups corresponding to the hemoglobin concentration data. The processing parameters include slicing parameters for the gray matter volume data, first sliding window parameters for the EEG data, and second sliding window parameters for the hemoglobin concentration data. The target processing operations include slicing operations for the gray matter volume data, post-sliding window normalization operations for the EEG data, and post-sliding window normalization operations for the hemoglobin concentration data. Each data sequence, each first sliding window data group, and each first sliding window data group has corresponding time-series parameters. Based on all brain region nodes in the preset brain region template, perform node mapping operations on all data sequences, all first sliding window data groups, and all second sliding window data groups to obtain a set of data sequences, a set of first sliding window data groups, and a set of second sliding window data groups corresponding to each brain region node. Based on the data sequence set corresponding to all brain region nodes, the corresponding first sliding window data set set, and the corresponding second sliding window data set set, a dynamic covariance matrix matching the brain multimodal data is determined, and the target weight parameters in the dynamic covariance matrix are adjusted to obtain the adjusted dynamic covariance matrix; the target weight parameters include a first weight parameter, a second weight parameter, and a third weight parameter. Based on the adjusted dynamic covariance matrix, the user's brain dynamic map object is determined; The dynamic covariance matrix is: w(t) = α*Cov(MRI,EEG)+β*Cov(EEG,fNIRS)+γ*Cov(MRI,fNIRS); α is the first weight parameter, β is the second weight parameter, γ is the third weight parameter, Cov(MRI, EEG) is the covariance parameter between all the data sequences and all the first sliding window data groups corresponding to the time parameter t, Cov(EEG, fNIRS) is the covariance parameter between all the first sliding window data groups and all the second sliding window data groups corresponding to the time parameter t, and Cov(MRI, fNIRS) is the covariance parameter between all the data sequences and all the second sliding window data groups corresponding to the time parameter t.

[0008] As an optional implementation, in the first aspect of the present invention, adjusting the target weight parameters in the dynamic covariance matrix to obtain an adjusted dynamic covariance matrix includes: Obtain intervention parameters that match the brain multimodal data; the intervention parameters include pharmacological intervention parameters and / or non-pharmacological intervention parameters; The module parameters of the preset learning module and the computational power consumption increment parameters that match the brain multimodal data are determined, and the reward parameters of the learning module are determined based on the module parameters and the computational power consumption increment parameters; the module parameters of the learning module include classification accuracy improvement parameters and module confidence parameters; Based on the intervention parameters and the reward parameters of the learning module, the weight adjustment parameters of the target weight parameters in the dynamic covariance matrix are determined, and the target weight parameters are adjusted according to the weight adjustment parameters to obtain the adjusted dynamic covariance matrix. The reward parameter for the learning module is as follows: r(t) = △Acc + n*L + m*J; △Acc is the classification accuracy improvement parameter, n is the preset confidence balance coefficient, L is the module confidence parameter, m is the preset power consumption weight parameter, and △Power is the calculation power consumption increment parameter.

[0009] As an optional implementation, in the first aspect of the present invention, the step of performing a feature fusion operation on the brain motion map object to obtain the user's brain multimodal feature fusion result includes: Based on the brain dynamic map object, a graph attention function for a target modality matching the brain dynamic map object and original feature parameters corresponding to each brain region node are determined. The target modality includes MRI modality, EEG modality, and hemoglobin modality. The graph attention function under the MRI modality, the graph attention function under the EEG modality, and the graph attention function under the hemoglobin modality are respectively used to calculate the interaction feature parameters between all brain region nodes within the MRI modality, the EEG modality, and the hemoglobin modality. The original feature parameters corresponding to each brain region node include the original feature parameters corresponding to the brain region node under the MRI modality, the EEG modality, and the hemoglobin modality. Based on the graph attention function of the target modality and the original feature parameters corresponding to each brain region node, the fused feature parameters of each brain region node are determined, and based on the fused feature parameters of all brain region nodes, the multimodal feature fusion result of the user's brain is determined. The fused feature parameters of the corresponding brain region node i are: ; m is an index parameter for the target modality, used to indicate the MRI modality, the EEG modality, or the hemoglobin modality. m For the preset attention weight parameters in the m-th modality, GAF m Let f be the graph attention function for the m-th mode. im f represents the original feature parameters corresponding to brain region node i in the m-th modality. jm Let be the original feature parameters corresponding to brain region node j in the m-th modality.

[0010] As an optional implementation, in a first aspect of the present invention, determining the user's brain function monitoring data based on the brain multimodal feature fusion result includes: Based on the brain multimodal feature fusion results, the functional connectivity strength parameters between any two brain region nodes are calculated, and based on the functional connectivity strength parameters between any two brain region nodes, the path length parameters corresponding to any two brain region nodes are determined. Based on the path length parameters corresponding to each pair of brain region nodes and the number of nodes corresponding to all brain region nodes, determine the whole-brain graph theory index parameters corresponding to all brain region nodes; Based on the dimension parameters of the brain multimodal feature fusion result, a vector concatenation operation is performed on the brain multimodal feature fusion result and the whole brain graph theory index parameters to obtain an extended feature vector; The extended feature vector is input into a preset monitoring module for analysis, and the analysis results of the monitoring module are used as the user's brain function monitoring data.

[0011] A second aspect of this invention discloses an intelligent brain function monitoring system based on multimodal data, the system comprising: The acquisition module is used to acquire multimodal brain data of the user through a preset head-worn device; the head-worn device integrates an MRI scanner, an EEG parameter acquisition device, and a hemoglobin parameter acquisition device, and the multimodal brain data includes gray matter volume data, EEG data, and hemoglobin concentration data; The determination module is used to determine the user's brain motion map object based on the brain multimodal data; The feature fusion module is used to perform feature fusion operations on the brain dynamic map object based on the brain dynamic map object to obtain the user's brain multimodal feature fusion result; The determining module is further configured to determine the user's brain function monitoring data based on the brain multimodal feature fusion results.

[0012] As an optional implementation, in a second aspect of the invention, the determining module is further configured to: Before the acquisition module acquires the user's multimodal brain data through the preset head-worn device, the pulse test parameters for the EEG parameter acquisition device in the head-worn device are determined based on the gradient magnetic field parameters of the MRI scanner in the preset head-worn device. The system also includes: The measurement module is used to measure the interference parameters of the electrode pairs of the EEG parameter acquisition device according to the pulse test parameters; The determining module is further configured to determine the pulse adjustment parameters of the EEG parameter acquisition device based on the interference parameters of the electrode pair; An adjustment module is used to adjust the initial pulse parameters of the EEG parameter acquisition device according to the pulse adjustment parameters, so as to obtain the adjusted pulse parameters of the EEG parameter acquisition device. Specifically, the acquisition module collects multimodal brain data from the user through a pre-set head-worn device in the following ways: Multimodal brain data of the user is collected through a pre-set head-worn device, based on the adjusted pulse parameters and the pre-set first acquisition control parameters of the MRI scanner, the second acquisition control parameters of the EEG parameter collector, and the third acquisition control parameters of the hemoglobin parameter collector.

[0013] As an optional implementation, in a second aspect of the invention, the system further includes: The acquisition module acquires a first target parameter of the EEG parameter acquisition device and a second target parameter of the magnetic resonance imaging (MRI) device during the acquisition process of the user's EEG data by the acquisition module. The first target parameter of the EEG parameter acquisition device includes the shielding coefficient of the EEG parameter acquisition device, the parameters of the effective EEG signal acquired, and the parameters of the basic noise interference received. The second target parameter of the MRI device includes the magnetic field strength parameter and the gradient switching interval parameter of the MRI device. The determining module is further configured to determine the signal-to-noise ratio parameter of the EEG signal of the EEG parameter acquisition device based on the first target parameter and the second target parameter; A filtering module is used to filter the EEG data according to the EEG signal-to-noise ratio parameter in order to update the EEG data; The signal-to-noise ratio parameter of the EEG signal is: SNR_EEG = 20*log10(V_signal / (V_noise+k*B_MRI*Δt); V_signal is the effective EEG signal parameter, V_noise is the basic noise interference parameter, k is the shielding coefficient, B_MRI is the magnetic field strength parameter, and Δt is the gradient switching interval parameter.

[0014] As an optional implementation, in a second aspect of the present invention, the method by which the determining module determines the user's brain motion map object based on the brain multimodal data specifically includes: According to preset processing parameters, target processing operations are performed on the brain multimodal data to obtain multiple data sequences corresponding to the gray matter volume data, multiple first sliding window data groups corresponding to the EEG data, and multiple second sliding window data groups corresponding to the hemoglobin concentration data. The processing parameters include slicing parameters for the gray matter volume data, first sliding window parameters for the EEG data, and second sliding window parameters for the hemoglobin concentration data. The target processing operations include slicing operations for the gray matter volume data, post-sliding window normalization operations for the EEG data, and post-sliding window normalization operations for the hemoglobin concentration data. Each data sequence, each first sliding window data group, and each first sliding window data group has corresponding time-series parameters. Based on all brain region nodes in the preset brain region template, perform node mapping operations on all data sequences, all first sliding window data groups, and all second sliding window data groups to obtain a set of data sequences, a set of first sliding window data groups, and a set of second sliding window data groups corresponding to each brain region node. Based on the data sequence set corresponding to all brain region nodes, the corresponding first sliding window data set set, and the corresponding second sliding window data set set, a dynamic covariance matrix matching the brain multimodal data is determined, and the target weight parameters in the dynamic covariance matrix are adjusted to obtain the adjusted dynamic covariance matrix; the target weight parameters include a first weight parameter, a second weight parameter, and a third weight parameter. Based on the adjusted dynamic covariance matrix, the user's brain dynamic map object is determined; The dynamic covariance matrix is: w(t) = α*Cov(MRI,EEG)+β*Cov(EEG,fNIRS)+γ*Cov(MRI,fNIRS); α is the first weight parameter, β is the second weight parameter, γ is the third weight parameter, Cov(MRI, EEG) is the covariance parameter between all the data sequences and all the first sliding window data groups corresponding to the time parameter t, Cov(EEG, fNIRS) is the covariance parameter between all the first sliding window data groups and all the second sliding window data groups corresponding to the time parameter t, and Cov(MRI, fNIRS) is the covariance parameter between all the data sequences and all the second sliding window data groups corresponding to the time parameter t.

[0015] As an optional implementation, in a second aspect of the present invention, the method by which the determining module adjusts the target weight parameters in the dynamic covariance matrix to obtain the adjusted dynamic covariance matrix specifically includes: Obtain intervention parameters that match the brain multimodal data; the intervention parameters include pharmacological intervention parameters and / or non-pharmacological intervention parameters; The module parameters of the preset learning module and the computational power consumption increment parameters that match the brain multimodal data are determined, and the reward parameters of the learning module are determined based on the module parameters and the computational power consumption increment parameters; the module parameters of the learning module include classification accuracy improvement parameters and module confidence parameters; Based on the intervention parameters and the reward parameters of the learning module, the weight adjustment parameters of the target weight parameters in the dynamic covariance matrix are determined, and the target weight parameters are adjusted according to the weight adjustment parameters to obtain the adjusted dynamic covariance matrix. The reward parameter for the learning module is as follows: r(t) = △Acc + n*L + m*J; △Acc is the classification accuracy improvement parameter, n is the preset confidence balance coefficient, L is the module confidence parameter, m is the preset power consumption weight parameter, and △Power is the calculation power consumption increment parameter.

[0016] As an optional implementation, in a second aspect of the present invention, the feature fusion module performs feature fusion operations on the brain motion map object based on the brain motion map object to obtain the user's brain multimodal feature fusion result, specifically including: Based on the brain dynamic map object, a graph attention function for a target modality matching the brain dynamic map object and original feature parameters corresponding to each brain region node are determined. The target modality includes MRI modality, EEG modality, and hemoglobin modality. The graph attention function under the MRI modality, the graph attention function under the EEG modality, and the graph attention function under the hemoglobin modality are respectively used to calculate the interaction feature parameters between all brain region nodes within the MRI modality, the EEG modality, and the hemoglobin modality. The original feature parameters corresponding to each brain region node include the original feature parameters corresponding to the brain region node under the MRI modality, the EEG modality, and the hemoglobin modality. Based on the graph attention function of the target modality and the original feature parameters corresponding to each brain region node, the fused feature parameters of each brain region node are determined, and based on the fused feature parameters of all brain region nodes, the multimodal feature fusion result of the user's brain is determined. The fused feature parameters of the corresponding brain region node i are: ; m is an index parameter for the target modality, used to indicate the MRI modality, the EEG modality, or the hemoglobin modality. m For the preset attention weight parameters in the m-th modality, GAF m Let f be the graph attention function for the m-th mode. im f represents the original feature parameters corresponding to brain region node i in the m-th modality. jm Let be the original feature parameters corresponding to brain region node j in the m-th modality.

[0017] As an optional implementation, in a second aspect of the present invention, the method by which the determining module determines the user's brain function monitoring data based on the brain multimodal feature fusion result specifically includes: Based on the brain multimodal feature fusion results, the functional connectivity strength parameters between any two brain region nodes are calculated, and based on the functional connectivity strength parameters between any two brain region nodes, the path length parameters corresponding to any two brain region nodes are determined. Based on the path length parameters corresponding to each pair of brain region nodes and the number of nodes corresponding to all brain region nodes, determine the whole-brain graph theory index parameters corresponding to all brain region nodes; Based on the dimension parameters of the brain multimodal feature fusion result, a vector concatenation operation is performed on the brain multimodal feature fusion result and the whole brain graph theory index parameters to obtain an extended feature vector; The extended feature vector is input into a preset monitoring module for analysis, and the analysis results of the monitoring module are used as the user's brain function monitoring data.

[0018] A third aspect of this invention discloses another intelligent brain function monitoring system based on multimodal data, the system comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the intelligent brain function monitoring method based on multimodal data disclosed in the first aspect of the present invention.

[0019] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute the intelligent brain function monitoring method based on multimodal data disclosed in the first aspect of the present invention.

[0020] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: In this embodiment of the invention, a wearable head device is used to collect multimodal brain data from the user. Based on the multimodal brain data, a dynamic brain image of the user is identified, and feature fusion is performed on the dynamic brain image to obtain the user's multimodal brain feature fusion result. Based on the multimodal brain feature fusion result, the user's brain function monitoring data is determined. Therefore, implementing this invention allows for the collection and intelligent analysis of multimodal brain data from a user using a portable wearable head device, thereby determining the user's brain function monitoring data. This not only reduces reliance on large brain monitoring equipment but also accurately reflects the dynamic changes in the user's brain function through multimodal data, thus improving the reliability and accuracy of monitoring the user's brain function data. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating an intelligent brain function monitoring method based on multimodal data disclosed in an embodiment of the present invention. Figure 2 This is a flowchart illustrating another intelligent brain function monitoring method based on multimodal data disclosed in an embodiment of the present invention; Figure 3This is a schematic diagram of the structure of an intelligent brain function monitoring system based on multimodal data disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram of another intelligent brain function monitoring system based on multimodal data disclosed in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of another intelligent brain function monitoring system based on multimodal data disclosed in an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0025] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0026] This invention discloses a method and system for intelligent monitoring of brain function based on multimodal data. It not only reduces the reliance on large-scale brain monitoring equipment, but also accurately reflects the dynamic changes in the user's brain function through multimodal data, thereby improving the reliability and accuracy of monitoring the user's brain function data.

[0027] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating an intelligent brain function monitoring method based on multimodal data, as disclosed in an embodiment of the present invention. Wherein, Figure 1The described intelligent brain function monitoring method based on multimodal data can be applied to monitor the function of various brain regions, such as the entorhinal cortex, cerebral cortex, hippocampus, posterior cingulate cortex, prefrontal cortex, basal forebrain, etc., and is not limited in the embodiments of this invention. Optionally, this method can be implemented by a brain function monitoring system, which can be integrated into a smart terminal device (such as a smart computer, smartphone, tablet, etc.), or it can be a local server or cloud server used to process the intelligent brain function monitoring process based on multimodal data, etc., and is not limited in the embodiments of this invention. Figure 1 As shown, this intelligent brain function monitoring method based on multimodal data may include the following operations: 101. Collect multimodal brain data from users through a pre-set head-worn device.

[0028] In this embodiment of the invention, the head-worn device integrates a magnetic resonance imaging (MRI) scanner, an electroencephalogram (EEG) parameter acquisition device, and a hemoglobin parameter acquisition device. For example, the head-worn device integrates a low-field portable magnetic resonance imaging (MRI) scanner, a dry electrode EEG parameter acquisition headband (EEG headband), and a near-infrared (fNIRS) sensor, thereby achieving simultaneous acquisition of whole-brain structural (gray matter volume), functional (EEG power spectrum), and metabolic (oxygenation signal) data.

[0029] Furthermore, the brain multimodal data includes gray matter volume data, electroencephalogram (EEG) data, and hemoglobin concentration data.

[0030] Step 101 can be understood to include the following sub-steps: Sub-step 1011: The user starts the terminal, and the head-worn device initializes and performs a self-test.

[0031] After the user presses the power button on the terminal, the main control unit starts a self-test program: checking the temperature of the low-field MRI magnet; calibrating the contact impedance of the EEG dry electrode; verifying the stability of the fNIRS light source, etc. After the self-test passes, a wearable guidance animation pops up on the main unit display screen.

[0032] Step 1012: The user wears the head-mounted device and initiates low-field MRI scan, EEG measurement, and fNIRS measurement.

[0033] Step 1013: Simultaneously trigger the acquisition of three data channels. The present invention does not limit the scanning time, sampling rate, window length, data cache format, etc.

[0034] 102. Based on the brain multimodal data, determine the user's brain dynamic map object, and perform feature fusion operation on the brain dynamic map object to obtain the user's brain multimodal feature fusion result.

[0035] In this embodiment of the invention, it can be understood as using gray matter volume data acquired by low-field portable MRI to construct a whole-brain 3D structural model, using EEG power spectrum acquired by EEG headband to label functionally active regions, and combining hemoglobin concentration changes acquired by fNIRS sensor to map metabolic activity hotspots to generate a spatiotemporal dynamic brain atlas. Then, multimodal feature fusion operation is performed on the spatiotemporal dynamic brain atlas to obtain data results that fuse MRI modality, EEG modality, and fNIRS modality features.

[0036] 103. Based on the fusion results of brain multimodal features, determine the user's brain function monitoring data.

[0037] In this embodiment of the invention, it can be understood as establishing a dynamic correlation model of structure-function-metabolic parameters through the fusion results of multimodal features, analyzing the synergistic change law between brain region activity patterns and multidimensional physiological signals, and then generating quantitative monitoring indicators of user brain functions (such as cognitive function, motor control function, visuospatial function, language function, etc.), which is beneficial for subsequent related scholars to conduct risk research on brain function damage / degeneration.

[0038] It should be noted that this invention achieves on-site deployment and continuous monitoring through three major innovations: "lightweight hardware + real-time edge intelligence + enhanced adaptive learning". Specifically, by integrating a dynamic multimodal network analysis engine and an adaptive learning module, it supports real-time monitoring of the brain function of the monitored person in scenarios such as home and community. Moreover, the dynamic network weights can be adjusted in real time according to the state of the monitored person through reinforcement learning algorithms.

[0039] Furthermore, after identifying the user's brain function monitoring data, reports (such as changes in brain function, explainable causal chains, brain heatmaps, etc.) can be generated and sent to the user's smart terminal devices, such as smartphones, smart computers, and tablets, to support user viewing or printing. Additionally, brain function monitoring results can be visually presented to the user through voice broadcasts or LED displays. Moreover, the user's brain function monitoring data can be anonymized before transmission.

[0040] As can be seen, implementing the embodiments of the present invention enables the collection and intelligent analysis of users' multimodal brain data through portable head-worn devices, thereby determining users' brain function monitoring data. This not only reduces reliance on large brain monitoring equipment, but also accurately reflects the dynamic changes in users' brain function through multimodal data, thereby improving the reliability and accuracy of monitoring users' brain function data.

[0041] Example 2 Please see Figure 2 , Figure 2This is a flowchart illustrating another intelligent brain function monitoring method based on multimodal data disclosed in an embodiment of the present invention. Wherein, Figure 2 The described intelligent brain function monitoring method based on multimodal data can be applied to monitor the function of various brain regions, such as the entorhinal cortex, cerebral cortex, hippocampus, posterior cingulate cortex, prefrontal cortex, basal forebrain, etc., and is not limited in the embodiments of this invention. Optionally, this method can be implemented by a brain function monitoring system, which can be integrated into a smart terminal device (such as a smart computer, smartphone, tablet, etc.), or it can be a local server or cloud server used to process the intelligent brain function monitoring process based on multimodal data, etc., and is not limited in the embodiments of this invention. Figure 2 As shown, this intelligent brain function monitoring method based on multimodal data may include the following operations: 201. Based on the preset gradient magnetic field parameters of the MRI scanner in the head-worn device, determine the pulse test parameters for the EEG parameter acquisition device in the head-worn device.

[0042] In this embodiment of the invention, optionally, the pulse test parameters include at least one of the following: pulse amplitude test parameters, pulse frequency test parameters, pulse waveform test parameters, and pulse time interval test parameters.

[0043] 202. Based on the pulse test parameters, measure the interference parameters of the electrode pairs of the EEG parameter acquisition device, and determine the pulse adjustment parameters of the EEG parameter acquisition device based on the interference parameters of the electrode pairs.

[0044] In this embodiment of the invention, optionally, the interference parameters of the electrode pair include at least one of the voltage interference parameters, current interference parameters, and magnetic field interference parameters.

[0045] Further optionally, the pulse adjustment parameters include at least one of the following: pulse amplitude adjustment parameters, pulse frequency adjustment parameters, pulse waveform adjustment parameters, pulse time interval adjustment parameters, and pulse delay adjustment parameters.

[0046] 203. Adjust the initial pulse parameters of the EEG parameter acquisition device according to the pulse adjustment parameters to obtain the adjusted pulse parameters of the EEG parameter acquisition device.

[0047] In this embodiment of the invention, the EEG parameter acquisition device needs to be calibrated before MRI scanning. The interference phase difference of each electrode pair of the EEG parameter acquisition device can be measured by short-time test pulses to dynamically adjust its anti-phase pulse parameters.

[0048] It should be noted that, due to electromagnetic interference (EMI) and data asynchrony issues between MRI and EEG equipment in traditional monitoring technologies, a multi-layer copper mesh shielded cavity design can be adopted. Combined with MRI gradient coil timing inversion technology, the shielded cavity effectively attenuates low-frequency interference (gradient switching). Specifically, the rapidly changing magnetic field (dB / dt) generated during MRI gradient coil switching induces eddy currents in the EEG electrodes and leads, forming common-mode interference (CMI). The copper mesh shielded cavity, through the Faraday cage effect, can confine the external electromagnetic field outside the shielding layer, while simultaneously suppressing internal circuitry radiating interference to the outside. Furthermore, timing inversion can suppress high-frequency harmonic effects (especially spatially related interference components), improving the EEG signal-to-noise ratio.

[0049] Furthermore, the MRI gradient magnetic field varies linearly in space, and the interference voltage generated on the EEG electrodes is related to the electrode position. By controlling the timing of the gradient coils, the interference voltages of adjacent electrode pairs can be made out of phase, canceling each other out upon superposition. That is, a tiny anti-phase pulse is superimposed on a traditional gradient waveform (such as a trapezoidal wave), adjusting its amplitude and phase to make the interference voltage of the target EEG electrode pair out of phase. Example: If the interference voltages of electrodes a and b are Va = V0sin(ωt) and Vb = V0sin(ωt+φ) respectively, then by using an anti-phase pulse to make φ approximately equal to 180°, Va + Vb approximately equal to 0. The delay of the anti-phase pulse can be controlled in the nanosecond range to ensure phase matching. An FPGA (Field-Programmable Gate Array) can be used to generate the anti-phase pulse in real time, combined with a pre-computed lookup table (LUT) to reduce latency.

[0050] 204. The user's brain multimodal data is collected through a preset head-worn device and according to the adjusted pulse parameters and the preset first acquisition control parameters of the MRI scanner, the second acquisition control parameters of the EEG parameter collector, and the third acquisition control parameters of the hemoglobin parameter collector.

[0051] In this embodiment of the invention, optionally, the first acquisition control parameter, the second acquisition control parameter, and the third acquisition control parameter may each include at least one of the following: acquisition frequency parameter, acquisition duration parameter, acquisition start and end time parameter.

[0052] 205. Based on the brain multimodal data, determine the user's brain dynamic map object, and perform feature fusion operation on the brain dynamic map object to obtain the user's brain multimodal feature fusion result.

[0053] 206. Based on the fusion results of brain multimodal features, determine the user's brain function monitoring data.

[0054] In this embodiment of the invention, for other descriptions of steps 205 and 206, please refer to the detailed description of steps 102 and 103 in Embodiment 1. These descriptions will not be repeated in this embodiment of the invention.

[0055] As can be seen, implementing the embodiments of the present invention can effectively solve the eddy current interference problem caused by rapid switching of gradient magnetic fields in traditional MRI-EEG synchronous acquisition systems by using pulse dynamic calibration technology for EEG acquisition devices based on MRI gradient magnetic field parameters, combined with a composite anti-interference design of multi-layer copper mesh shielded cavity and gradient coil timing reversal. Simultaneously, by superimposing fine-tuned reversible pulses into the traditional trapezoidal wave, the phase difference of interference voltage between adjacent electrode pairs of the EEG parameter acquisition device is precisely controlled within a preset range, achieving active cancellation of common-mode interference and thus effectively improving the signal-to-noise ratio of the EEG signal. Furthermore, by controlling the MRI scanner, EEG parameter acquisition device, and hemoglobin parameter acquisition device in the head-worn device with corresponding acquisition control parameters, the user's multimodal brain data can be acquired, which is beneficial to improving the real-time performance and accuracy of multimodal brain data acquisition.

[0056] In an optional embodiment, the method further includes: During the process of collecting users' EEG data, the first target parameters of the EEG parameter acquisition device and the second target parameters of the magnetic resonance imaging device are obtained. Based on the first target parameter and the second target parameter, the signal-to-noise ratio (SNR) parameter of the EEG signal acquisition device is determined, and the EEG data is filtered according to the SNR parameter to update the EEG data.

[0057] In this optional embodiment, an adaptive filtering algorithm (such as LMS) can be used to further optimize the aforementioned cancellation effect. The first target parameters of the EEG parameter acquisition device include the shielding coefficient of the EEG parameter acquisition device, the parameters of the acquired effective EEG signals, and the parameters of the underlying noise interference. The second target parameters of the MRI scanner include the magnetic field strength parameters and the gradient switching interval parameters of the MRI scanner.

[0058] Furthermore, the signal-to-noise ratio parameter of the EEG signal is: SNR_EEG = 20*log10(V_signal / (V_noise+k*B_MRI*Δt); Wherein, V_signal is the effective EEG signal parameter, V_noise is the basic noise interference parameter, k is the shielding coefficient, B_MRI is the magnetic field strength parameter, and Δt is the gradient switching interval parameter.

[0059] As can be seen, this optional embodiment can dynamically acquire the shielding coefficient, effective signal strength, and basic noise level of the EEG parameter acquisition device during the EEG data acquisition process, and combine this with the real-time magnetic field strength and gradient switching timing parameters of the MRI scanner to calculate the signal-to-noise ratio (SNR) parameter of the EEG signal. Then, based on the SNR parameter of the EEG signal, the EEG data can be filtered. This not only improves the accuracy and effectiveness of EEG data filtering, but also suppresses the common-mode noise caused by MRI gradient interference as much as possible while fully preserving the frequency band characteristics of the effective EEG signal. This is beneficial for dynamically maintaining the quality of the EEG signal in a strong magnetic field environment, and provides reliable signal quality assurance for high spatiotemporal resolution brain function monitoring data research.

[0060] In another optional embodiment, step 205 above, determining the user's brain motion map object based on brain multimodal data, includes: Based on preset processing parameters, target processing operations are performed on brain multimodal data to obtain multiple data sequences corresponding to gray matter volume data, multiple first sliding window data groups corresponding to EEG data, and multiple second sliding window data groups corresponding to hemoglobin concentration data. Based on all brain region nodes in the preset brain region template, perform node mapping operations on all data sequences, all first sliding window data groups, and all second sliding window data groups to obtain the data sequence set, the first sliding window data group set, and the second sliding window data group set corresponding to each brain region node. Based on the data sequence set corresponding to all brain region nodes, the corresponding first sliding window data set set, and the corresponding second sliding window data set set, a dynamic covariance matrix matching the brain multimodal data is determined, and the target weight parameters in the dynamic covariance matrix are adjusted to obtain the adjusted dynamic covariance matrix. Based on the adjusted dynamic covariance matrix, the user's brain dynamic map object is determined.

[0061] In this optional embodiment, it can be understood that various modal data are first time-sliced ​​and truncated using sliding windows. The multiple data sequences corresponding to the acquired gray matter volume data, the multiple first sliding window data groups corresponding to the EEG data, and the multiple second sliding window data groups corresponding to the hemoglobin concentration data are sent into a buffer to realize the cross-modal spatial registration process. For example, based on the AAL brain region template, the gray matter volume data is mapped to multiple brain region nodes, and the EEG data and hemoglobin concentration data are matched to the corresponding brain region nodes according to the electrode / probe position (i.e., affine transformation). Then, a unified node feature matrix (m brain region nodes * 3 modalities) is generated, and a covariance matrix is ​​dynamically generated to output a brain dynamic graph object containing the embedding of each brain region node and time-varying edge weights (edges represent dynamic covariance or functional connectivity).

[0062] For example, this cross-modal registration process may include the following steps: 1. Multi-scale time-slice algorithm: A signal processing method that decomposes time series data into slices of different time scales (time resolution) and analyzes them independently or collaboratively at each scale.

[0063] 2. Intra-scale features: Extract statistical features (such as mean, variance, frequency domain energy) or time series features (such as autocorrelation coefficient) at each time scale.

[0064] 3. Cross-scale association: Combining features from different scales through multi-scale fusion (such as weighted average, deep learning attention mechanism).

[0065] 4. Spatial registration protocol: An improved ANTs registration algorithm is adopted to uniformly map multimodal data to the AAL3 spectral space, thereby improving registration accuracy.

[0066] In this optional embodiment, the processing parameters further include slicing parameters for gray matter volume data (such as slicing interval parameters in years, total number of slices, etc.), first sliding window parameters for EEG data (such as sliding window size parameters in seconds), and second sliding window parameters for hemoglobin concentration data (such as sliding window size parameters in seconds). Furthermore, the target processing operations include slicing operations for gray matter volume data, post-sliding window standardization operations for EEG data, and post-sliding window standardization operations for hemoglobin concentration data; each data sequence, each first sliding window data group, and each first sliding window data group has corresponding time-series parameters; the target weight parameters include a first weight parameter, a second weight parameter, and a third weight parameter.

[0067] In this optional embodiment, the dynamic covariance matrix is ​​further defined as follows: w(t) = α*Cov(MRI,EEG)+β*Cov(EEG,fNIRS)+γ*Cov(MRI,fNIRS); Where α is the first weighting parameter, β is the second weighting parameter, γ is the third weighting parameter, Cov(MRI, EEG) is the covariance parameter between all data sequences and all first sliding window data groups under time parameter t, Cov(EEG, fNIRS) is the covariance parameter between all first sliding window data groups and all second sliding window data groups under time parameter t, and Cov(MRI, fNIRS) is the covariance parameter between all data sequences and all second sliding window data groups under time parameter t.

[0068] It should be noted that this formula can integrate the advantages of MRI, EEG, and fNIRS through weighted covariance to overcome the limitations of a single modality (such as the low temporal resolution of fMRI and the low spatial resolution of EEG). Here, α, β, and γ are weighting coefficients used to dynamically adjust the contribution ratio of different modalities. α emphasizes the synergistic effect of MRI and EEG, β focuses on the complementarity of EEG and fNIRS, and γ focuses on the complementarity of MRI and fNIRS. Dynamic adaptation is achieved through reinforcement learning (i.e., continuously adjusting weights, such as using GRU / Transformer units to iteratively update the features of each brain region node, inputting multi-time-point network tensors into a Temporal GNN, and outputting the optimal time-varying edge weights). The time window slides across the signal to analyze dynamic functional connectivity. The sliding window makes the weight W(t) change over time, which can be applied to studying the dynamic reorganization of brain networks in cognitive tasks.

[0069] In this optional embodiment, further, after obtaining the multiple data sequences corresponding to the gray matter volume data, the multiple first sliding window data groups corresponding to the EEG data, and the multiple second sliding window data groups corresponding to the hemoglobin concentration data (i.e., modal data slices), an interpretable causal weight can be assigned to each temporal edge within the framework of a dynamic Bayesian network (DBN), combining Granger causality tests and structural equation modeling (SEM). 1. Dynamic Bayesian Network (DBN) Modeling: Define variable nodes for time slices t and t+1, and construct temporal edges (e.g., Xt→Yt+1).

[0070] 2. Granger Causality Test: For each candidate edge X→Y, fit an autoregressive model (AR) to Yt+1. If the goodness of fit (e.g., R²) significantly improves after adding Xt, then X has Granger causality with Y.

[0071] 3. Structural Equation Modeling (SEM) Path Analysis: Edges that pass the Granger test are included in the SEM to estimate the path coefficient β. XY (Standardized regression weights) (when β) XY When X increases by 1 unit, Y is expected to increase by k units.

[0072] 4. Causal weight integration: final weight W XY =Granger p-value*β XY Where p-value reflects significance, β XY The magnitude of the effect.

[0073] As can be seen, this optional embodiment can use a multi-scale time-slicing algorithm to perform hierarchical analysis of gray matter volume, EEG, and hemoglobin concentration data. Combined with intra-scale feature extraction and subsequent cross-scale attention fusion mechanism, it can effectively capture the dynamic evolution of brain functional activity at the millisecond to interannual scales. At the same time, by using the ANTs registration algorithm to uniformly map multimodal data to the AAL3 brain region template, and combining a weighted fusion strategy based on dynamic covariance matrix, it breaks through the inherent limitations of single modality in spatiotemporal resolution. This organically integrates the structural stability of MRI, the temporal sensitivity of EEG, and the metabolic specificity of fNIRS, so as to optimize the contribution ratio and time-varying edge weights of the three modality pairs of MRI-EEG, EEG-fNIRS, and MRI-fNIRS in real time. This can accurately track the dynamic reorganization process of brain network topology during cognitive task execution. Furthermore, a dynamic Bayesian network framework is introduced, and through joint modeling of Granger causality test and structural equation model, interpretable causal semantics are given to the functional connectivity between brain regions. This enables the generated dynamic brain graph objects to not only contain node-level time-varying features, but also have edge-level causal reasoning capabilities. It provides a full-chain analysis tool from brain functional connectivity to causal mechanism analysis, which significantly enhances the clinical translational value of brain science research.

[0074] In another optional embodiment, the step of adjusting the target weight parameters in the dynamic covariance matrix to obtain the adjusted dynamic covariance matrix includes: Obtain intervention parameters that match brain multimodal data; Determine the module parameters of the preset learning module and the computational power consumption increment parameters that match the brain multimodal data, and determine the reward parameters of the learning module based on the module parameters and the computational power consumption increment parameters. Based on the intervention parameters and the reward parameters of the learning module, the weight adjustment parameters of the target weight parameters in the dynamic covariance matrix are determined, and the target weight parameters are adjusted according to the weight adjustment parameters to obtain the adjusted dynamic covariance matrix.

[0075] In this optional embodiment, the target weight parameters can be adjusted multiple times to iteratively optimize the multimodal fusion weights.

[0076] Optionally, intervention parameters include pharmacological intervention parameters (such as drug type, dosage, and administration time) and / or non-pharmacological intervention parameters (such as living environment parameters, user age, and user's physiological / psychological condition). Furthermore, the weight adjustment parameters for the target weight parameters in the dynamic covariance matrix can be determined by further combining the network topology characteristics of the learning module (such as average path length and node degree centrality) and dynamic time series characteristics (such as coherence / phase synchronization).

[0077] Furthermore, the module parameters of the learning module include classification accuracy improvement parameters and module confidence parameters.

[0078] Furthermore, the reward parameters for the learning module are: r(t) = △Acc + n*L + m*J; △Acc is the classification accuracy improvement parameter (where △Acc = Acc(t) - Acc(t-1)), n is the preset confidence balance coefficient, L is the module confidence parameter (which can be determined by the module's prediction probability distribution of the current sample), m is the preset power consumption weight parameter, and △Power is the power consumption increment parameter (where △Power = Power(t) - Power(t-1), and Power can be determined by GPU memory and FLOPs).

[0079] The setting of n can reduce the excessive pursuit of module confidence at the expense of accuracy.

[0080] As can be seen, this optional embodiment can dynamically couple intervention parameters with the reward function of the learning module, enabling the weight adjustment process to respond to changes in brain function caused by physiological intervention while also meeting the real-time requirements of the algorithm. Specifically, the introduction of a confidence balancing coefficient into the reward function effectively reduces the risk of overfitting caused by the module excessively pursuing high-confidence predictions, while the dynamic adjustment mechanism of the power consumption weight parameters ensures the deployability of the weight optimization process on mobile or embedded devices. Furthermore, the iterative optimization strategy supports multiple adjustments of the weight parameters, coupled with auxiliary constraints from network topology features and time-series features, allowing the generated dynamic covariance matrix to continuously approximate the optimal multimodal fusion state. This enables the constructed brain dynamic map object to accurately reflect the time-varying patterns of brain functional connectivity under intervention conditions and possesses strong robustness to individual differences, thus accurately analyzing changes in the user's brain function.

[0081] In another optional embodiment, step 205 above, which involves performing a feature fusion operation on the brain motion map object to obtain the user's multimodal brain feature fusion result, includes: Based on the brain dynamics map object, determine the graph attention function of the target modality that matches the brain dynamics map object and the original feature parameters corresponding to each brain region node; Based on the graph attention function of the target modality and the original feature parameters corresponding to each brain region node, the fused feature parameters of each brain region node are determined, and based on the fused feature parameters of all brain region nodes, the user's multimodal brain feature fusion result is determined.

[0082] In this optional embodiment, it can be understood that each modality feature is dynamically weighted at the node level according to the weights output by the policy network, and then aggregated into a final embedding vector through multi-head attention.

[0083] The target modalities include the MRI modality, EEG modality, and hemoglobin modality. Furthermore, the graph attention functions in the MRI modality, EEG modality, and hemoglobin modality are used to calculate the interaction feature parameters between all brain region nodes within the MRI modality, EEG modality, and hemoglobin modality, respectively, as well as the original feature parameters corresponding to each brain region node, including the original feature parameters of that brain region node in the MRI modality, EEG modality, and hemoglobin modality.

[0084] Furthermore, the fused feature parameters of the corresponding brain region node i are: ; m is the index parameter for the target modality, used to indicate the MRI modality, EEG modality, or hemoglobin modality (e.g., 1 for MRI modality, 2 for EEG modality, 3 for hemoglobin modality), W m For the preset attention weight parameters in the m-th modality, GAF m Let f be the graph attention function for the m-th mode. im f represents the original feature parameters corresponding to brain region node i in the m-th modality. jm Let be the original feature parameters corresponding to brain region node j in the m-th modality.

[0085] As can be seen, this optional embodiment can determine graph attention functions for the three modalities of MRI, EEG, and hemoglobin, enabling each brain region node to dynamically perceive the topological relationships of other nodes within the same modality. This allows for the capture of functional connectivity patterns within the modality while preserving the original modality-specific features. Furthermore, by introducing a joint regulation mechanism of modality index parameters and attention weight parameters, the original features of different modalities are adaptively weighted and fused at the node level, effectively solving the individual adaptability problem of fixed-weight fusion.

[0086] In another optional embodiment, step 206 above, determining the user's brain function monitoring data based on the brain multimodal feature fusion results, includes: Based on the fusion results of brain multimodal features, the functional connectivity strength parameters between pairs of brain region nodes are calculated, and the path length parameters corresponding to the pairs of brain region nodes are determined based on the functional connectivity strength parameters between the pairs of brain region nodes. Based on the path length parameters corresponding to each pair of brain region nodes and the number of nodes corresponding to all brain region nodes, determine the whole-brain graph theory index parameters corresponding to all brain region nodes. Based on the dimensional parameters of the brain multimodal feature fusion results, a vector concatenation operation is performed on the brain multimodal feature fusion results and the whole-brain graph theory index parameters to obtain an extended feature vector; The extended feature vector is input into a preset monitoring module for analysis, and the analysis results of the monitoring module are used as the user's brain function monitoring data.

[0087] In this optional embodiment, the whole-brain graph theory metric parameters may optionally include at least one of modularity parameters, global efficiency parameters, and clustering coefficients.

[0088] The brain multimodal feature fusion results include the multimodal feature fusion results of each brain region node. i The functional connectivity strength parameter between any two brain region nodes can be: W ij =corr(h i h j The path length parameter corresponding to each pair of brain region nodes can be: L ij =-log(W) ij The global efficiency parameter in the whole-brain graph theory index parameters can be: (N is the number of nodes), the extended feature vector can be: input=[h i ;E]∈R d+1 (d is the dimension parameter).

[0089] As can be seen, this optional embodiment can first calculate the functional connectivity strength between brain region nodes based on fusion features, and then determine the path length parameters corresponding to each pair of brain region nodes. This preserves the gradient information of the original correlation and enhances the discriminative power of long-distance connections. Furthermore, by combining quantitative analysis with graph theory indicators, it characterizes the whole-brain information integration capability and functional modularity level from the perspective of network topology, effectively overcoming the limitations of single connectivity strength analysis. At the same time, by dimensionally concatenating the fusion feature vector and graph theory indicators, the constructed extended feature vector not only includes the multimodal dynamic features of each brain region but also integrates the functional organization pattern of the whole-brain network. This allows the monitoring data to simultaneously reflect the microscopic state and macroscopic network characteristics of brain functional activities, thereby significantly improving the sensitivity and biological interpretability of brain function monitoring.

[0090] Example 3 Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an intelligent brain function monitoring system based on multimodal data, as disclosed in an embodiment of the present invention. Figure 3 As shown, this intelligent brain function monitoring system based on multimodal data may include: The acquisition module 301 is used to acquire multimodal brain data of the user through a preset head-worn device; The determination module 302 is used to determine the user's brain motion map object based on brain multimodal data; The feature fusion module 303 is used to perform feature fusion operations on the brain dynamic map object based on the brain dynamic map object to obtain the user's brain multimodal feature fusion result; The determination module 302 is also used to determine the user's brain function monitoring data based on the fusion results of brain multimodal features.

[0091] In this embodiment of the invention, the head-worn device integrates an MRI scanner, an EEG parameter acquisition device, and a hemoglobin parameter acquisition device. The multimodal brain data includes gray matter volume data, EEG data, and hemoglobin concentration data.

[0092] It is evident that implementation Figure 3 The described intelligent brain function monitoring system based on multimodal data can collect and intelligently analyze users' multimodal brain data through a portable head-worn device, thereby determining the user's brain function monitoring data. This not only reduces the reliance on large brain monitoring equipment, but also accurately reflects the dynamic changes in users' brain function through multimodal data, thus improving the reliability and accuracy of monitoring users' brain function data.

[0093] In an optional embodiment, the determining module 302 is further configured to: Before the acquisition module 301 acquires the user's multimodal brain data through the preset head-worn device, the pulse test parameters for the EEG parameter acquisition device in the head-worn device are determined based on the gradient magnetic field parameters of the MRI scanner in the preset head-worn device. The system also includes: Measurement module 304 is used to measure the interference parameters of the electrode pairs of the EEG parameter acquisition device according to the pulse test parameters; The determination module 302 is also used to determine the pulse adjustment parameters of the EEG parameter acquisition device based on the interference parameters of the electrode pair; The adjustment module 305 is used to adjust the initial pulse parameters of the EEG parameter acquisition device according to the pulse adjustment parameters, so as to obtain the adjusted pulse parameters of the EEG parameter acquisition device. Specifically, the acquisition module 301 acquires the user's multimodal brain data through a pre-set head-worn device in the following ways: The system collects multimodal brain data from the user through a pre-set head-worn device, based on adjusted pulse parameters and pre-set first acquisition control parameters of the MRI scanner, second acquisition control parameters of the EEG scanner, and third acquisition control parameters of the hemoglobin scanner.

[0094] It is evident that implementation Figure 4 The described intelligent brain function monitoring system based on multimodal data effectively solves the eddy current interference problem caused by rapid switching of gradient magnetic fields in traditional MRI-EEG synchronous acquisition systems. This is achieved through pulse dynamic calibration technology for EEG acquisition devices based on MRI gradient magnetic field parameters, combined with a composite anti-interference design of multi-layer copper mesh shielded cavity and gradient coil timing reversal. Furthermore, by superimposing fine-tuned reversible pulses onto the traditional trapezoidal wave, the phase difference of interference voltage between adjacent electrode pairs of the EEG parameter acquisition device is precisely controlled within a preset range, achieving active cancellation of common-mode interference and thus effectively improving the signal-to-noise ratio of the EEG signal. In addition, by controlling the MRI scanner, EEG parameter acquisition device, and hemoglobin parameter acquisition device in the wearable head device with corresponding acquisition control parameters, the system can acquire the user's multimodal brain data, improving the real-time performance and accuracy of multimodal brain data acquisition.

[0095] In another alternative embodiment, the system further includes: The acquisition module 306 acquires the first target parameter of the EEG parameter acquisition device and the second target parameter of the magnetic resonance imaging device during the process of the acquisition module 301 acquiring the user's EEG data. The determining module 302 is also used to determine the signal-to-noise ratio parameter of the EEG signal of the EEG parameter acquisition device based on the first target parameter and the second target parameter; The filtering module 307 is used to filter the EEG data according to the EEG signal-to-noise ratio parameter in order to update the EEG data.

[0096] In this optional embodiment, the first target parameters of the EEG parameter acquisition device include the shielding coefficient of the EEG parameter acquisition device, the parameters of the effective EEG signals acquired, and the parameters of the basic noise interference received. The second target parameters of the MRI scanner include the magnetic field strength parameters of the MRI scanner and the gradient switching interval parameters.

[0097] Furthermore, the signal-to-noise ratio parameter of the EEG signal is: SNR_EEG = 20*log10(V_signal / (V_noise+k*B_MRI*Δt); V_signal is the effective EEG signal parameter, V_noise is the basic noise interference parameter, k is the shielding coefficient, B_MRI is the magnetic field strength parameter, and Δt is the gradient switching interval parameter.

[0098] It is evident that implementation Figure 4The described intelligent brain function monitoring system based on multimodal data can dynamically acquire the shielding coefficient, effective signal strength, and basic noise level of the EEG parameter acquisition device during the EEG data acquisition process. Combined with the real-time magnetic field strength and gradient switching timing parameters of the MRI scanner, it calculates the signal-to-noise ratio (SNR) of the EEG signal. Then, based on the SNR of the EEG signal, it performs filtering operations on the EEG data. This not only improves the accuracy and effectiveness of EEG data filtering, but also suppresses common-mode noise caused by MRI gradient interference as much as possible while fully preserving the frequency band characteristics of the effective EEG signal. This is beneficial for dynamically maintaining the quality of EEG signals in a strong magnetic field environment, providing reliable signal quality assurance for high spatiotemporal resolution brain function monitoring data research.

[0099] In yet another optional embodiment, the method by which the determining module 302 determines the user's brain motion map object based on brain multimodal data specifically includes: Based on preset processing parameters, target processing operations are performed on brain multimodal data to obtain multiple data sequences corresponding to gray matter volume data, multiple first sliding window data groups corresponding to EEG data, and multiple second sliding window data groups corresponding to hemoglobin concentration data. Based on all brain region nodes in the preset brain region template, perform node mapping operations on all data sequences, all first sliding window data groups, and all second sliding window data groups to obtain the data sequence set, the first sliding window data group set, and the second sliding window data group set corresponding to each brain region node. Based on the data sequence set corresponding to all brain region nodes, the corresponding first sliding window data set set, and the corresponding second sliding window data set set, a dynamic covariance matrix matching the brain multimodal data is determined, and the target weight parameters in the dynamic covariance matrix are adjusted to obtain the adjusted dynamic covariance matrix. Based on the adjusted dynamic covariance matrix, the user's brain dynamic map object is determined.

[0100] In this optional embodiment, the processing parameters include slicing parameters for gray matter volume data, first sliding window parameters for EEG data, and second sliding window parameters for hemoglobin concentration data; the target processing operations include slicing operations for gray matter volume data, post-sliding window normalization operations for EEG data, and post-sliding window normalization operations for hemoglobin concentration data; each data sequence, each first sliding window data group, and each first sliding window data group has corresponding time-series parameters. The target weight parameters include a first weight parameter, a second weight parameter, and a third weight parameter.

[0101] Furthermore, the dynamic covariance matrix is: w(t) = α*Cov(MRI,EEG)+β*Cov(EEG,fNIRS)+γ*Cov(MRI,fNIRS); α is the first weighting parameter, β is the second weighting parameter, γ is the third weighting parameter, Cov(MRI, EEG) is the covariance parameter between all data sequences and all first sliding window data groups under time parameter t, Cov(EEG, fNIRS) is the covariance parameter between all first sliding window data groups and all second sliding window data groups under time parameter t, and Cov(MRI, fNIRS) is the covariance parameter between all data sequences and all second sliding window data groups under time parameter t.

[0102] It is evident that implementation Figure 4 The described intelligent brain function monitoring system based on multimodal data can perform hierarchical analysis of gray matter volume, EEG, and hemoglobin concentration data using a multi-scale time-slice algorithm. Combined with intra-scale feature extraction and subsequent cross-scale attention fusion mechanism, it can effectively capture the dynamic evolution of brain functional activity at the millisecond to interannual scales. Simultaneously, by using the ANTs registration algorithm to uniformly map multimodal data to the AAL3 brain region template, and combining a weighted fusion strategy based on dynamic covariance matrix, it overcomes the inherent limitations of single modality in spatiotemporal resolution. This organically integrates the structural stability of MRI, the temporal sensitivity of EEG, and the metabolic specificity of fNIRS, thereby optimizing the contribution ratio and time-varying edge weights of the three modality pairs of MRI-EEG, EEG-fNIRS, and MRI-fNIRS in real time. This allows for precise tracking of the dynamic reorganization process of brain network topology during cognitive task execution. Furthermore, a dynamic Bayesian network framework is introduced, and through joint modeling of Granger causality test and structural equation model, interpretable causal semantics are given to the functional connectivity between brain regions. This enables the generated dynamic brain graph objects to not only contain node-level time-varying features, but also have edge-level causal reasoning capabilities. It provides a full-chain analysis tool from brain functional connectivity to causal mechanism analysis, which significantly enhances the clinical translational value of brain science research.

[0103] In another optional embodiment, the method by which the determining module 302 adjusts the target weight parameters in the dynamic covariance matrix to obtain the adjusted dynamic covariance matrix specifically includes: Obtain intervention parameters that match brain multimodal data; Determine the module parameters of the preset learning module and the computational power consumption increment parameters that match the brain multimodal data, and determine the reward parameters of the learning module based on the module parameters and the computational power consumption increment parameters. Based on the intervention parameters and the reward parameters of the learning module, the weight adjustment parameters of the target weight parameters in the dynamic covariance matrix are determined, and the target weight parameters are adjusted according to the weight adjustment parameters to obtain the adjusted dynamic covariance matrix.

[0104] In this optional embodiment, the intervention parameters include drug intervention parameters and / or non-drug intervention parameters; the module parameters of the learning module include classification accuracy improvement parameters and module confidence parameters.

[0105] Furthermore, the reward parameters for the learning module are: r(t) = △Acc + n*L + m*J; △Acc is the classification accuracy improvement parameter, n is the preset confidence balance coefficient, L is the module confidence parameter, m is the preset power consumption weight parameter, and △Power is the power consumption increment parameter.

[0106] It is evident that implementation Figure 4 The described intelligent brain function monitoring system based on multimodal data dynamically couples intervention parameters with the reward function of the learning module. This allows the weight adjustment process to respond to changes in brain function caused by physiological intervention while also meeting the real-time requirements of the algorithm. Specifically, the introduction of a confidence balancing coefficient into the reward function effectively reduces the risk of overfitting caused by the module excessively pursuing high-confidence predictions. The dynamic adjustment mechanism of the power consumption weight parameters ensures the deployability of the weight optimization process on mobile or embedded devices. Furthermore, an iterative optimization strategy supports multiple adjustments of the weight parameters. Combined with auxiliary constraints from network topology features and time-series features, the generated dynamic covariance matrix continuously approximates the optimal multimodal fusion state. This allows the constructed brain dynamic map object to accurately reflect the time-varying patterns of brain functional connectivity under intervention conditions and possesses strong robustness to individual differences, enabling precise analysis of changes in user brain function.

[0107] In another optional embodiment, the feature fusion module 303 performs feature fusion operations on the brain motion map object based on the brain motion map object to obtain the user's brain multimodal feature fusion result in the following specific ways: Based on the brain dynamics map object, determine the graph attention function of the target modality that matches the brain dynamics map object and the original feature parameters corresponding to each brain region node; Based on the graph attention function of the target modality and the original feature parameters corresponding to each brain region node, the fused feature parameters of each brain region node are determined, and based on the fused feature parameters of all brain region nodes, the user's multimodal brain feature fusion result is determined.

[0108] In this optional embodiment, the target modality includes the MRI modality, the EEG modality, and the hemoglobin modality. The graph attention function in the MRI modality, the graph attention function in the EEG modality, and the graph attention function in the hemoglobin modality are respectively used to calculate the interaction feature parameters between all brain region nodes in the MRI modality, the EEG modality, and the hemoglobin modality. The original feature parameters corresponding to each brain region node include the original feature parameters corresponding to that brain region node in the MRI modality, the EEG modality, and the hemoglobin modality.

[0109] Furthermore, the fused feature parameters of the corresponding brain region node i are: ; m is the index parameter for the target modality, used to indicate the MRI modality, EEG modality, or hemoglobin modality. W m For the preset attention weight parameters in the m-th modality, GAF m Let f be the graph attention function for the m-th mode. im f represents the original feature parameters corresponding to brain region node i in the m-th modality. jm Let be the original feature parameters corresponding to brain region node j in the m-th modality.

[0110] It is evident that implementation Figure 4 The described intelligent brain function monitoring system based on multimodal data can determine graph attention functions for three modalities: MRI, EEG, and hemoglobin. This allows each brain region node to dynamically perceive the topological relationships of other nodes within the same modality, thereby capturing functional connectivity patterns within the modality while preserving the original modality-specific features. Furthermore, by introducing a joint regulation mechanism of modality index parameters and attention weight parameters, the original features of different modalities are adaptively weighted and fused at the node level, effectively solving the individual adaptability problem of fixed-weight fusion.

[0111] In yet another optional embodiment, the method by which the determining module 302 determines the user's brain function monitoring data based on the brain multimodal feature fusion results specifically includes: Based on the fusion results of brain multimodal features, the functional connectivity strength parameters between pairs of brain region nodes are calculated, and the path length parameters corresponding to the pairs of brain region nodes are determined based on the functional connectivity strength parameters between the pairs of brain region nodes. Based on the path length parameters corresponding to each pair of brain region nodes and the number of nodes corresponding to all brain region nodes, determine the whole-brain graph theory index parameters corresponding to all brain region nodes. Based on the dimensional parameters of the brain multimodal feature fusion results, a vector concatenation operation is performed on the brain multimodal feature fusion results and the whole-brain graph theory index parameters to obtain an extended feature vector; The extended feature vector is input into a preset monitoring module for analysis, and the analysis results of the monitoring module are used as the user's brain function monitoring data.

[0112] It is evident that implementation Figure 4 The described intelligent brain function monitoring system based on multimodal data first calculates the functional connectivity strength between brain region nodes based on fused features, then determines the path length parameters corresponding to each pair of brain region nodes. This preserves the gradient information of the original correlations and enhances the discriminative power of long-distance connections. Furthermore, by combining quantitative analysis with graph theory indicators, it characterizes the whole-brain information integration capability and functional modularity level from the perspective of network topology, effectively overcoming the limitations of single connectivity strength analysis. Simultaneously, by dimensionally concatenating the fused feature vector and graph theory indicators, the constructed extended feature vector includes both the multimodal dynamic features of each brain region and integrates the functional organization pattern of the whole-brain network. This allows the monitoring data to simultaneously reflect the microscopic state and macroscopic network characteristics of brain functional activity, thereby significantly improving the sensitivity and biological interpretability of brain function monitoring.

[0113] Example 4 Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of another intelligent brain function monitoring system based on multimodal data disclosed in an embodiment of the present invention. Figure 5 As shown, this intelligent brain function monitoring system based on multimodal data may include: Memory 401 storing executable program code; Processor 402 coupled to memory 401; The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the intelligent brain function monitoring method based on multimodal data described in Embodiment 1 or Embodiment 2 of the present invention.

[0114] Example 5 This invention discloses a computer storage medium storing computer instructions. When these computer instructions are invoked, they are used to execute the steps in the intelligent brain function monitoring method based on multimodal data described in Embodiment 1 or Embodiment 2 of this invention.

[0115] Example 6 This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the intelligent brain function monitoring method based on multimodal data described in Embodiment 1 or Embodiment 2.

[0116] The system embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0117] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0118] Finally, it should be noted that the intelligent brain function monitoring method and system based on multimodal data disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent monitoring of brain function based on multimodal data, characterized in that, The method includes: The system collects multimodal brain data from users through a pre-set head-worn device. The head-worn device integrates an MRI scanner, an EEG parameter acquisition device, and a hemoglobin parameter acquisition device. The multimodal brain data includes gray matter volume data, EEG data, and hemoglobin concentration data. Based on the brain multimodal data, the user's brain motion map object is determined, and based on the brain motion map object, a feature fusion operation is performed on the brain motion map object to obtain the user's brain multimodal feature fusion result; Based on the results of the fusion of the brain's multimodal features, the user's brain function monitoring data is determined.

2. The intelligent brain function monitoring method based on multimodal data according to claim 1, characterized in that, Before collecting the user's multimodal brain data through a preset head-worn device, the method further includes: Based on the preset gradient magnetic field parameters of the MRI scanner in the head-worn device, determine the pulse test parameters for the EEG parameter acquisition device in the head-worn device. Based on the pulse test parameters, the interference parameters of the electrode pairs of the EEG parameter acquisition device are measured, and the pulse adjustment parameters of the EEG parameter acquisition device are determined based on the interference parameters of the electrode pairs. Based on the pulse adjustment parameters, the initial pulse parameters of the EEG parameter acquisition device are adjusted to obtain the adjusted pulse parameters of the EEG parameter acquisition device; The process of collecting multimodal brain data from a user via a pre-set head-worn device includes: Multimodal brain data of the user is collected through a pre-set head-worn device, based on the adjusted pulse parameters and the pre-set first acquisition control parameters of the MRI scanner, the second acquisition control parameters of the EEG parameter collector, and the third acquisition control parameters of the hemoglobin parameter collector.

3. The intelligent brain function monitoring method based on multimodal data according to claim 2, characterized in that, The method further includes: During the process of collecting the user's EEG data, the first target parameters of the EEG parameter acquisition device and the second target parameters of the magnetic resonance imaging (MRI) device are obtained. The first target parameters of the EEG parameter acquisition device include the shielding coefficient of the EEG parameter acquisition device, the parameters of the effective EEG signals collected, and the parameters of the basic noise interference. The second target parameters of the MRI device include the magnetic field strength parameters and the gradient switching interval parameters of the MRI device. Based on the first target parameter and the second target parameter, the EEG signal-to-noise ratio parameter of the EEG parameter acquisition device is determined, and the EEG data is filtered based on the EEG signal-to-noise ratio parameter to update the EEG data; The signal-to-noise ratio parameter of the EEG signal is: SNR_EEG = 20*log10(V_signal / (V_noise+k*B_MRI*Δt); V_signal is the effective EEG signal parameter, V_noise is the basic noise interference parameter, k is the shielding coefficient, B_MRI is the magnetic field strength parameter, and Δt is the gradient switching interval parameter.

4. The intelligent brain function monitoring method based on multimodal data according to any one of claims 1-3, characterized in that, The step of determining the user's brain motion map object based on the brain multimodal data includes: According to preset processing parameters, target processing operations are performed on the brain multimodal data to obtain multiple data sequences corresponding to the gray matter volume data, multiple first sliding window data groups corresponding to the EEG data, and multiple second sliding window data groups corresponding to the hemoglobin concentration data. The processing parameters include slicing parameters for the gray matter volume data, first sliding window parameters for the EEG data, and second sliding window parameters for the hemoglobin concentration data. The target processing operations include slicing operations for the gray matter volume data, post-sliding window normalization operations for the EEG data, and post-sliding window normalization operations for the hemoglobin concentration data. Each data sequence, each first sliding window data group, and each first sliding window data group has corresponding time-series parameters. Based on all brain region nodes in the preset brain region template, perform node mapping operations on all data sequences, all first sliding window data groups, and all second sliding window data groups to obtain a set of data sequences, a set of first sliding window data groups, and a set of second sliding window data groups corresponding to each brain region node. Based on the data sequence set corresponding to all brain region nodes, the corresponding first sliding window data set set, and the corresponding second sliding window data set set, a dynamic covariance matrix matching the brain multimodal data is determined, and the target weight parameters in the dynamic covariance matrix are adjusted to obtain the adjusted dynamic covariance matrix; the target weight parameters include a first weight parameter, a second weight parameter, and a third weight parameter. Based on the adjusted dynamic covariance matrix, the user's brain dynamic map object is determined; The dynamic covariant matrix is: w(t) = α*Cov(MRI,EEG)+β*Cov(EEG,fNIRS)+γ*Cov(MRI,fNIRS); α is the first weight parameter, β is the second weight parameter, γ is the third weight parameter, Cov(MRI, EEG) is the covariance parameter between all the data sequences and all the first sliding window data groups corresponding to the time parameter t, Cov(EEG, fNIRS) is the covariance parameter between all the first sliding window data groups and all the second sliding window data groups corresponding to the time parameter t, and Cov(MRI, fNIRS) is the covariance parameter between all the data sequences and all the second sliding window data groups corresponding to the time parameter t.

5. The intelligent brain function monitoring method based on multimodal data according to claim 4, characterized in that, The step of adjusting the target weight parameters in the dynamic covariance matrix to obtain the adjusted dynamic covariance matrix includes: Obtain intervention parameters that match the brain multimodal data; the intervention parameters include pharmacological intervention parameters and / or non-pharmacological intervention parameters; The module parameters of the preset learning module and the computational power consumption increment parameters that match the brain multimodal data are determined, and the reward parameters of the learning module are determined based on the module parameters and the computational power consumption increment parameters; the module parameters of the learning module include classification accuracy improvement parameters and module confidence parameters; Based on the intervention parameters and the reward parameters of the learning module, the weight adjustment parameters of the target weight parameters in the dynamic covariance matrix are determined, and the target weight parameters are adjusted according to the weight adjustment parameters to obtain the adjusted dynamic covariance matrix. The reward parameter for the learning module is as follows: r(t) = △Acc + n*L + m*J; △Acc is the classification accuracy improvement parameter, n is the preset confidence balance coefficient, L is the module confidence parameter, m is the preset power consumption weight parameter, and △Power is the calculation power consumption increment parameter.

6. The intelligent brain function monitoring method based on multimodal data according to claim 4, characterized in that, The step of performing feature fusion on the brain motion map object to obtain the user's multimodal brain feature fusion result includes: Based on the brain dynamic map object, a graph attention function for a target modality matching the brain dynamic map object and original feature parameters corresponding to each brain region node are determined. The target modality includes MRI modality, EEG modality, and hemoglobin modality. The graph attention function under the MRI modality, the graph attention function under the EEG modality, and the graph attention function under the hemoglobin modality are respectively used to calculate the interaction feature parameters between all brain region nodes within the MRI modality, the EEG modality, and the hemoglobin modality. The original feature parameters corresponding to each brain region node include the original feature parameters corresponding to the brain region node under the MRI modality, the EEG modality, and the hemoglobin modality. Based on the graph attention function of the target modality and the original feature parameters corresponding to each brain region node, the fused feature parameters of each brain region node are determined, and based on the fused feature parameters of all brain region nodes, the fusion result of the user's brain multimodal features is determined. The fused feature parameters of the corresponding brain region node i are: ； m is an index parameter for the target modality, used to indicate the MRI modality, the EEG modality, or the hemoglobin modality. m For the preset attention weight parameters in the m-th modality, GAF m Let f be the graph attention function for the m-th mode. im f represents the original feature parameters corresponding to brain region node i in the m-th modality. jm Let be the original feature parameters corresponding to brain region node j in the m-th modality.

7. The intelligent brain function monitoring method based on multimodal data according to claim 6, characterized in that, The step of determining the user's brain function monitoring data based on the brain multimodal feature fusion results includes: Based on the brain multimodal feature fusion results, the functional connectivity strength parameters between any two brain region nodes are calculated, and based on the functional connectivity strength parameters between any two brain region nodes, the path length parameters corresponding to any two brain region nodes are determined. Based on the path length parameters corresponding to each pair of brain region nodes and the number of nodes corresponding to all brain region nodes, determine the whole-brain graph theory index parameters corresponding to all brain region nodes; Based on the dimension parameters of the brain multimodal feature fusion result, a vector concatenation operation is performed on the brain multimodal feature fusion result and the whole brain graph theory index parameters to obtain an extended feature vector; The extended feature vector is input into a preset monitoring module for analysis, and the analysis results of the monitoring module are used as the user's brain function monitoring data.

8. A brain function intelligent monitoring system based on multimodal data, characterized in that, The system includes: The acquisition module is used to acquire multimodal brain data of the user through a preset head-worn device; the head-worn device integrates an MRI scanner, an EEG parameter acquisition device, and a hemoglobin parameter acquisition device, and the multimodal brain data includes gray matter volume data, EEG data, and hemoglobin concentration data; The determination module is used to determine the user's brain motion map object based on the brain multimodal data; The feature fusion module is used to perform feature fusion operations on the brain dynamic map object based on the brain dynamic map object to obtain the user's brain multimodal feature fusion result; The determining module is further configured to determine the user's brain function monitoring data based on the brain multimodal feature fusion results.

9. A brain function intelligent monitoring system based on multimodal data, characterized in that, The system includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the intelligent brain function monitoring method based on multimodal data as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked, are used to execute the intelligent brain function monitoring method based on multimodal data as described in any one of claims 1-7.

Images