A method and device for causal analysis of brain imaging signals for asynchronous EEG-fNIRS

Abstract

The present invention belongs to the field of electroencephalogram (EEG) signal analysis, and specifically relates to a method and device for causal analysis of brain imaging signals for asynchronous EEG-fNIRS, including: creating a psychological experiment paradigm for guiding patients to perform cognitive ability tests; collecting and preprocessing asynchronous EEG and fNIRS signals generated by patients during the execution of the psychological experiment paradigm; extracting spatial features from the fNIRS signals to process the EEG signals to align them spatially; extracting temporal features from the EEG signals to process the fNIRS signals to align them temporally; and analyzing the causal relationship between the EEG and fNIRS signals after spatial and temporal alignment. The present invention realizes the full utilization of spatio-temporal information of multiple modality signals collected by asynchronous non-union devices, and its flexibility and accuracy can support in-depth analysis of brain region states, providing an effective tool for neurovascular coupling analysis and clinical application of EEG and fNIRS.

Description

Technical Field

[0001] The present invention belongs to the field of electroencephalogram signal analysis, and particularly relates to a method and device for causal analysis of brain imaging signals for asynchronous EEG-fNIRS. Background Technique

[0002] Neurovascular Coupling (NVC) refers to the mutual relationship between neuronal activation and blood flow and metabolism in the corresponding regions. This mechanism is jointly participated by neurons, glial cells and vascular cells, revealing the close connection between neural electrical activity and hemodynamic changes. Under the action of external stimuli, the activation of neurons not only generates neural electrical activity, but also promotes blood flow to the active brain regions to meet their increased metabolic demands. This process is reflected in the change of blood oxygen level, specifically manifested as an increase in the concentration of oxyhemoglobin (HbO2) and a decrease in the concentration of deoxyhemoglobin (HbR). In recent years, NVC analysis has become a research hotspot in the field of brain science, providing a theoretical basis for revealing the association between brain neural activity and hemodynamics. Using multimodal brain imaging technology is one of the important directions in the field of NVC research. Analyzing the causal relationship of multimodal brain imaging signals helps to explain the deep state information and disease sources of the human brain system. Therefore, it is crucial to develop a systematic and reliable method for analyzing the causal relationship of brain imaging signals.

[0003] In the research on the method for analyzing the causal relationship of multimodal brain imaging signals, electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are two of the most commonly used neuroimaging techniques. EEG can provide information with high temporal resolution by measuring the potential changes on the scalp surface, and is suitable for capturing rapid neural electrical activity. However, the spatial resolution of EEG is limited and it cannot capture the data deep in the cortex. While fNIRS uses near-infrared light with different wavelengths of 650 - 950 nm to measure the changes in the concentrations of HbO2 and HbR in brain tissue. Although the temporal resolution is limited, it can provide relatively high spatial resolution. The respective unique advantages of EEG and fNIRS, and their complementarity in temporal and spatial characteristics make them powerful tools for studying the NVC mechanism and can deeply explore the brain neural activity under different cognitive tasks. However, most of the existing traditional neurovascular coupling studies have poor effects in describing the causal relationship of EEG-fNIRS signals and have limitations in the utilization rate of spatio-temporal signals in brain regions.

[0004] As disclosed in the Chinese patent application with the publication number CN118626786A, a method for causal analysis of spatio-temporal multi-scale electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) signals is disclosed. This method obtains the causal relationship of multi-modal signals based on EEG-fNIRS signals at different time scales, synthesizes the causal relationship through a weighted summation method, and obtains the results of causal analysis.

[0005] For another example, as disclosed in the Chinese patent application with the publication number CN117474103A, a method for cross-modal causal relationship analysis based on EEG-fNIRS is disclosed. This method uses variational mode decomposition and short-time Fourier transform to process multi-modal signals and completes the analysis of neurovascular coupling in different characteristic frequency bands based on causal strength values.

[0006] These previous inventions mainly focus on EEG-fNIRS signals under synchronous acquisition. On the one hand, due to the limitation of using EEG-fNIRS combined acquisition equipment, the spatio-temporal resolution of the two modal signals is relatively insufficient; on the other hand, for asynchronous EEG-fNIRS signals, the spatio-temporal information of the two modal signals is not fully utilized, and there is a lack of a standardized and reliable framework, making it difficult to be used in clinical applications. Summary of the Invention

[0007] In view of this, the present invention proposes a method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS. This method uses spatio-temporal alignment technology to extract and utilize the spatio-temporal information of the two modalities of asynchronous EEG-fNIRS signals to analyze the spatio-temporal causal relationship of the aligned data; it not only gets rid of the dependence on equipment in traditional multi-modal data analysis but also improves the flexibility and accuracy of causal analysis of brain imaging signals.

[0008] The technical solution of the present invention is as follows:

[0009] A method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS, comprising the following steps:

[0010] Step 1: Create a psychological experiment paradigm for guiding patients to perform cognitive ability tests;

[0011] Step 2: Collect two-modal data of asynchronous EEG signals and fNIRS signals generated by patients during the execution of the psychological experiment paradigm, and assign labels to the collected EEG signals and fNIRS signals;

[0012] Step 3: Signal preprocessing, respectively perform channel selection, filtering, epochs segmentation, downsampling, and hemoglobin concentration conversion processing on the collected EEG signals and fNIRS signals;

[0013] Step 4: By calculating the oxyhemoglobin concentration data in the fNIRS signal, using the method of screening EEG signal spatial channels based on the activation regions determined by the hemoglobin concentration distribution, spatially align the preprocessed EEG signal and fNIRS signal; based on the EEG signal, determine the temporal position of neuron activation, and then use the improved classical GLM method to process the fNIRS signal to temporally align the preprocessed EEG signal and fNIRS signal.

[0014] Step 5: After the EEG signal and fNIRS signal are spatially and temporally aligned, respectively perform empirical mode decomposition (EMD) on the EEG signal and fNIRS signal to obtain their respective main factor modal information; based on the main factor modal information of the EEG signal and the main factor modal information of the fNIRS signal, use convergent cross-mapping to solve and obtain the causal relationship between the EEG signal and the fNIRS signal.

[0015] Further, in the said Step 4, the process of spatially aligning the preprocessed EEG signal and fNIRS signal includes the steps:

[0016] Process the fNIRS signal using the modified Lambert-Beer law to obtain the multi-channel oxyhemoglobin concentration , where represents the number of divisions of a single experimental trial; then extract the activity coefficients of all channels, that is, the brain region activity information, through the improved common spatial pattern algorithm; select the spatial multi-dimensional channels that make up the maximum activity information to obtain the activity degree distribution of different brain regions; its calculation formula is as follows:

[0017] ;

[0018] ;

[0019] Among them, represents different activity degrees, represents the source signal common to the activity degrees of different channels, represents 's common spatial pattern, represents 's common spatial pattern, represents the maximum activity information of the spatial multi-dimensional channels, is the subspace combination of the spatial channels;

[0020] Based on the brain region activity information provided by the fNIRS signal and the activity degree distribution of different brain regions, screen the EEG signal channels of different brain regions to achieve the spatial alignment of the EEG signal and the fNIRS signal.

[0021] Further, the process of screening EEG signal channels of different brain regions based on the brain region activation information provided by fNIRS signals and the activation degree distribution of different brain regions includes:

[0022] Using the Brodmann area system, the brain region is subdivided into 52 refined partitions. Based on this, the positions of each channel of the fNIRS signal are corresponded to the Brodmann areas one by one, and the partition numbers corresponding to each active channel area are identified;

[0023] Within the range of the active brain region, all EEG signal channels and fNIRS signal channels adjacent to the specified partition numbers are screened out, and finally the alignment of EEG signals and fNIRS signals at the spatial level is achieved.

[0024] Further, in step 4, the process of time alignment of the preprocessed EEG signals and fNIRS signals includes the following steps:

[0025] The spatially aligned EEG signals are segmented into multiple time-window EEG time series segments by using an overlapping sliding window , and each time series segment contains an activation time series, where , represents the number of divisions of a single experimental trial;

[0026] The average power values of the EEG signals in the four frequency bands of Delta, Theta, Alpha, and Beta are extracted for each time window to obtain the multi-band power value change state in time series;

[0027] A linear predictor is established based on the multi-band power value change state, and a generalized functional linear model is established in combination with the fNIRS signal, as described below:

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] where is the fNIRS time series after time alignment, is the linear predictor, and are the response variables is the expected value of, is related to the natural parameter, is the cumulant generating function specific to the fNIRS distribution, is a normalization term specific to the fNIRS distribution, used to ensure that the integral value of the probability density function is 1, is the dispersion parameter of the model, are all polynomials composed of the changing states of multi-band power values over time, serving as predictor variables in the model, is the constant term of the model, are all vectors of regression coefficients to be estimated, representing the influence of each predictor variable on the response variable, is the number of samples of the EEG signal, is the link function of the generalized function linear model, is the likelihood function used to determine the parameter values;

[0033] Use the generalized function linear model to re-model the cerebral hemodynamic response on the time series of EEG signal activation, and complete the time alignment of the EEG signal and the fNIRS signal.

[0034] Furthermore, after the spatial alignment and time alignment of the EEG signal and the fNIRS signal are completed in step 5, the process of respectively applying empirical mode decomposition (EMD) to the EEG signal and the fNIRS signal to obtain their respective main factor modal information includes:

[0035] For the EEG signal: Construct a sliding time window to gradually read the temporal potential values of the EEG signal to find all its local maxima and minima; then use the spline interpolation method to construct the upper envelope and the lower envelope for the maxima and minima respectively, and remove the mean of the envelope from the EEG signal to obtain the intrinsic mode function components, that is, the main factor modal information;

[0036] The process of obtaining the main factor modal information of the fNIRS signal using empirical mode decomposition is the same as the process of obtaining the main factor modal information of the EEG signal using empirical mode decomposition.

[0037] Furthermore,

[0038] The process of using convergent cross-mapping to solve based on the main factor modal information of the EEG signal and the main factor modal information of the fNIRS signal to obtain the causal relationship between the EEG signal and the fNIRS signal includes:

[0039] For the intrinsic mode function components decomposed from the EEG signal and the fNIRS signal, perform convergent cross-mapping on each component, and sum them with weights to obtain the Pearson coefficient representing the causal relationship between the EEG signal and the fNIRS signal, and thus the causal relationship between the EEG signal and the fNIRS signal can be obtained; its calculation formula is as follows:

[0040] ;

[0041] ;

[0042] ;

[0043] wherein, is the result after state embedding of the EEG signal, m is the embedding dimension, is the delay time, i is the embedding index, is the regression result of predicting the fNIRS signal , is the regression coefficient, j is the number of delays used in the regression, is the historical state vector of the first to j time points before the index used in the regression, is the covariance, is the regression result is the standard deviation of all data values in , is the standard deviation of all data values in the fNIRS signal .

[0044] A brain imaging signal causal analysis device for asynchronous EEG-fNIRS includes: an experimental paradigm module, a data acquisition module, a data preprocessing module, a time alignment module, and a causal analysis module; the above-mentioned method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS is implemented through the above-mentioned modules.

[0045] A method and device for causal analysis of brain imaging signals for asynchronous EEG-fNIRS provided by the present invention, aiming at EEG signals and fNIRS signals with differences in both time and space under asynchronous acquisition, utilizes the characteristic that the spatial resolution of fNIRS signals is higher than that of EEG signals, extracts spatial features from fNIRS signals to process EEG signals, so as to align the channel regions of the two. By utilizing the millisecond-level time resolution characteristic of EG signals, the transient electroencephalogram oscillation characteristics can be captured, temporal features are extracted from EEG signals and used to process fNIRS signals, so as to align the time sequences of the two, and then based on empirical mode decomposition (EMD), the main factor modal information of each is obtained; according to the main factor modal information of EEG signals and the main factor modal information of fNIRS signals, convergent cross mapping is used to solve, and the causal relationship between EEG signals and fNIRS signals is obtained, thereby realizing the causal relationship analysis of multi-modal brain imaging data under asynchronous acquisition. The present invention realizes the full utilization of spatio-temporal information of multiple modal signals collected by asynchronous non-union devices, and its flexibility and accuracy can support in-depth analysis of brain region states, providing an effective tool for neurovascular coupling analysis and clinical applications of EEG and fNIRS. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1Flowchart of the method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS in the embodiment;

[0047] Figure 2 N-back paradigm diagram adopted in the embodiment;

[0048] Figure 3 Technical roadmap for causal relationship analysis of EEG signals and fNIRS signals in the embodiment

[0049] Figure 4 EEG and fNIRS diagrams with single-trial temporal alignment in the embodiment. Detailed implementation manners

[0050] The present invention will be described in detail below with reference to the accompanying drawings and embodiments, but the embodiments of the present invention are not limited thereto.

[0051] A method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS provided in this embodiment, as Figure 1 shown, includes the following steps:

[0052] Step 1, create an experimental paradigm:

[0053] The experimental paradigm module is used to present a guiding paradigm to the subject to guide the subject to activate brain regions related to cognitive functions through task behaviors. The N-back, a psychological experimental paradigm for working memory ability test, is adopted in this embodiment, and this paradigm is made by E-PRIME software, as Figure 2 shown. This paradigm is based on the classical psychological task N-back experiment and tests the attention and memory levels of patients. The experimental paradigm is divided into three groups: 0-back, 1-back, and 2-back according to the difficulty. The participant sits in front of the computer and observes the sequence of numbers or letters presented on the screen. In each task, each letter is presented for 0.5 seconds, followed by a blank screen lasting for 2 seconds until the next letter appears. The participant needs to judge as quickly and accurately as possible whether the currently presented number or letter is the same as the number or letter n positions before. If it is the same, press the "Yes" key on the keyboard; if it is different, press the "No" key on the keyboard. The duration of a single experimental trial is 2 seconds, including a cue time of 0.5 seconds and a rest time of 1.5 seconds. The E-PRIME experimental paradigm includes three rounds, each round has 3 blocks, and each block contains 15 experimental trials.

[0054] Step 2, signal acquisition:

[0055] The data acquisition module is used to collect asynchronous EEG signals and fNIRS signal data generated by the subject during the execution of the psychological experiment paradigm, transmit the signal data to the corresponding data acquisition software, label the signal data, and store it in the system.

[0056] In this embodiment, a near-infrared brain imaging device with the function of collecting fNIRS signals in the fronto-parietal-temporal lobe region is used to collect fNIRS signals. The device includes 1 multi-channel head-mounted device and 1 terminal installed with fNIRS signal acquisition software. The EEG signals are collected by an eegoTM mylab 32-channel electroencephalogram acquisition device, which mainly consists of 1 32-channel electroencephalogram cap, 1 32-channel 16kHz amplifier, and 1 terminal notebook equipped with eego64 electroencephalogram signal acquisition software. The acquisition and processing software can present data in real time and perform signal processing, and also has an event marking function. The head-mounted device is connected to the notebook wirelessly, and the collected EEG signals and fNIRS signals are transmitted to the terminal notebook in real time.

[0057] Step 3: Data preprocessing:

[0058] Read the signal data of the two modalities of EEG signals and fNIRS signals stored in the system in step 2, and perform preprocessing including channel selection, filtering, downsampling, epochs segmentation, hemoglobin concentration conversion, etc., and store the preprocessed EEG signals and fNIRS signals in the system. Among them:

[0059] The filtering operation is used to filter out noise interferences such as motion artifacts, eye movements, and heartbeats in the signals, while retaining the EEG signals and fNIRS signals in the target frequency band. Specifically, the EEG signal data is processed using a sixth-order Butterworth band-pass filter (bidirectional zero-phase filtering), and the passband frequency range is 1 Hz to 30 Hz; the fNIRS signals are filtered through a discrete cosine transform filter, and its effective frequency band is 0.001 Hz to 0.5 Hz.

[0060] The downsampling operation is used to reduce the length of the collected EEG signals and fNIRS signal data.

[0061] In the epochs segmentation stage, the EEG signals and fNIRS signal data are divided into independent trials of different experimental times according to the marking points during the acquisition process in step 2. For the fNIRS signals, the data within 0 - 30 seconds after the task block marking point is intercepted as a single experimental segment; for the EEG signals, the data within 0 - 2 seconds after each trial marking point is intercepted to form a single-trial EEG segment.

[0062] The hemoglobin concentration conversion operation is used to convert the change in infrared light intensity continuously collected by the fNIRS device into the relative concentration changes of oxyhemoglobin (HbO) and deoxyhemoglobin (HbR). Through the modified Lambert-Beer law, the attenuation change of light intensity at different wavelengths is calculated, and combined with the differential pathlength factor (DPF) and differential absorption coefficient, the dynamic change value of hemoglobin concentration is finally solved. The calculation formula is as follows:

[0063] ;

[0064] where d is the distance between the light source and the detector; and are the extinction coefficients of oxyhemoglobin concentration and deoxyhemoglobin at the corresponding wavelength ; is the change in optical density at wavelength after the sampling time; is the wavelength of the differential pathlength factor (i = 0, 1); and are the relative changes in oxyhemoglobin concentration and deoxyhemoglobin concentration respectively; is the time interval between two adjacent samplings.

[0065] It should be noted that the hemoglobin concentration conversion method in this embodiment is applicable to various hemoglobin data types such as oxyhemoglobin concentration and deoxyhemoglobin concentration. To more clearly elaborate on the technical solution of this embodiment, the fNIRS data involved in the subsequent description specifically refers to the oxyhemoglobin (HbO) concentration data obtained based on the fNIRS technology.

[0066] Step 4, spatio-temporal alignment and causal analysis:

[0067] Perform spatial alignment and temporal alignment on the preprocessed EEG signal and fNIRS signal, and then perform causal relationship analysis on the aligned EEG signal and fNIRS signal. The route for performing spatial alignment, temporal alignment, and causal analysis on the EEG signal and fNIRS signal in this embodiment is as Figure 3 shown:

[0068] This embodiment screens the spatial channels of the EEG signal to align the two-modal data based on the activation region obtained from the hemoglobin concentration distribution; the method of spatial alignment aims to extract features from the fNIRS signal with high-level spatial resolution and use them to process the EEG signal to align the channel regions of the two. This method is based on the activation region determined by the hemoglobin concentration distribution, screens the spatial channels of the EEG signal, and realizes the spatial alignment of the two-modal data. The specific steps include:

[0069] The fNIRS signals are processed using the modified Lambert-Beer law to obtain the concentration of oxyhemoglobin in multiple channels , where represents the number of trial divisions in a single experiment; then, the active coefficients of all channels, i.e., the brain region activation information, are extracted by improving the common spatial pattern (CSP) algorithm; and the spatial multi-dimensional channels that make up the maximum active information are selected to obtain the activation degree distribution of different brain regions. The calculation formulas are shown in Formulas (2) and (3):

[0070] ;

[0071] ;

[0072] Among them, represents different activation degrees, represents the source signal common to the activation degrees of different channels, represents 's common spatial pattern, represents 's common spatial pattern, represents the maximum active information of the spatial multi-dimensional channels, is the subspace combination of the spatial channels.

[0073] Through the distribution information of the activation degrees of different brain regions, multi-layer screening is performed on the channels of the EEG signals. Since the EEG signals follow the international 10-20 channel standard and the channel positions are fixed; while the number of channels of the fNIRS signals is more dense and the channel positions are relatively flexible. To achieve multi-layer screening of the EEG signal channels, in this embodiment, based on the brain region activation information provided by the fNIRS signals and combined with the activation degree distribution of different brain regions, the EEG signal channels of different brain regions are screened. Specifically: Using the Brodmann area system, the brain regions are subdivided into 52 refined partitions. Based on this, the positions of each channel of the fNIRS signals are corresponded to the Brodmann areas one by one, and the partition numbers corresponding to each active channel region are identified. Within the range of the active brain regions, all the EEG signal channels and fNIRS signal channels adjacent to the specified partition numbers are screened out, and finally, the precise alignment of the EEG signals and fNIRS signals at the spatial level is achieved, giving full play to the advantage of the high spatial resolution of the fNIRS.

[0074] A method for time alignment aims to extract features from high-time-resolution EEG signals and use them to process fNIRS signals to align the time series of the two. This method is based on the temporal position of neuron activation obtained from EEG signals, and then uses an improved classical GLM method to process fNIRS signals to synchronize and align the time series of the two modal data. The specific steps include:

[0075] Set a sliding time window with a duration of 1000 ms and a window overlap rate of 50%, and divide the spatially aligned EEG signals into EEG time series segments of multiple time windows , where , represents the number of divisions of a single experimental trial. Subsequently, extract the average power values of the electroencephalogram (EEG) signals in four frequency bands of Delta (0.5 - 4 hz), Theta (4 - 8 hz), Alpha (8 - 13 hz), and Beta (13 - 30 Hz) under each time window to obtain the changing state of multi-band power values in time series , and its calculation formula is as shown in formula (4):

[0076] (4);

[0077] Among them, represents the complex representation of the i-th sub-segment of in the frequency domain f, obtained by Fourier transform, represents the modulus of , is the sum of the moduli of all sub-segments, is to further sum the sum of the modulus values corresponding to all frequencies;

[0078] Based on the changing state of multi-band power values, establish a linear predictor and combine it with fNIRS signals to establish a generalized function linear model, as shown in formulas (5), (6), (7), and (8); use the generalized function linear model to re-model the hemodynamic response on the time series of EEG signal activation to complete the time alignment of EEG signals and fNIRS signals. The single-trial time-aligned EEG signals and fNIRS signals are as Figure 4 shown:

[0079] (5);

[0080] (6);

[0081] (7);

[0082] (8);

[0083] Among them, is the fNIRS time series after time alignment, is the linear predictor, and is the response variable 's expected value, is related to the natural parameter, is the cumulant generating function specific to the fNIRS distribution, is the normalization term specific to the fNIRS distribution, used to ensure that the integral value of the probability density function is 1, is the dispersion parameter of the model, are all polynomials composed of the changing states of multi-band power values in time series, serving as predictor variables in the model, is the constant term of the model, are all vectors of regression coefficients to be estimated, representing the influence of each predictor variable on the response variable, is the number of samples of the EEG signal, is the link function of the generalized functional linear model, is the likelihood function used to determine the parameter values;

[0084] The method of causal analysis, which is used to conduct causal relationship analysis on the EEG signal and fNIRS signal after spatio-temporal alignment processing. This method is based on empirical mode decomposition (EMD) to obtain the main factor mode information and uses convergent cross mapping to solve for the causal relationship between the two mode data. The specific steps include:

[0085] Perform empirical mode decomposition on the EEG signal and fNIRS signal that have completed spatial alignment and time alignment respectively. That is, construct sliding time windows for the EEG signal and fNIRS signal respectively to gradually read the local maxima and minima found, and construct the upper envelope line and lower envelope line for the maxima and minima respectively through spline interpolation method. Remove the mean of the envelope line from the EEG signal and fNIRS signal to obtain the intrinsic mode function components, that is, the main factor mode information as shown in the formula:

[0086] (9);

[0087] (10);

[0088] (11);

[0089] Among them, represents the upper envelope line coordinates, represents the upper and lower envelope line coordinates, m is the envelope line mean, and X is the two mode signals, The intrinsic mode function component obtained by continuously decomposing X, r is the remaining component of X after removing the intrinsic mode function in each decomposition, and t represents the time step.

[0090] For the intrinsic mode function components decomposed from the two signals, perform convergent cross mapping component by component, and weighted summation to obtain the Pearson coefficient representing the causal relationship between the two modal signals of the EEG signal and the fNIRS signal, so as to obtain the causal relationship between the EEG signal and the fNIRS signal; its calculation formula is as shown in Formula (12) - Formula (14):

[0091] (12);

[0092] (13);

[0093] (14);

[0094] Among them, is the result after state embedding of the EEG signal m is the embedding dimension, is the delay time, i is the embedding index, is the regression result for predicting the fNIRS signal of, is the regression coefficient, j is the number of delays used in the regression, is the historical state vector of the first to j time points before the index used in the regression, is the covariance, is the regression result of the standard deviation of all data values in, is the fNIRS signal of the standard deviation of all data values in.

[0095] Based on the above method, this embodiment also provides a brain image signal causal analysis device for asynchronous EEG-fNIRS, and the device includes: an experimental paradigm module, a data acquisition module, a data preprocessing module, a time alignment module, and a causal analysis module that are connected in sequence; among them:

[0096] The experimental paradigm module is used to present a guiding paradigm to the patient, guiding the patient to activate the brain regions related to cognitive functions through task behaviors. The paradigm adopted is the psychological experiment paradigm N - back for cognitive ability testing. By setting experimental combinations with multiple difficulties, the attention and memory levels of the patient are tested, and the working memory brain regions of the patient are activated. Different difficulties will stimulate different mental workload levels of the patient;

[0097] The data acquisition module is used to acquire asynchronous EEG signals and fNIRS signals generated by the subject during the execution of the experimental paradigm, and assign labels to the EEG signals;

[0098] The data preprocessing module is used to receive the acquired EEG signals and fNIRS signals, and perform processing including filtering, epochs segmentation, downsampling, and hemoglobin concentration conversion on the EEG signals and fNIRS signals respectively;

[0099] The spatial alignment module calculates the oxyhemoglobin concentration data in the fNIRS signals, and uses the method of screening the spatial channels of the EEG signals based on the activation regions determined by the hemoglobin concentration distribution to perform spatial alignment on the preprocessed EEG signals and fNIRS signals;

[0100] The temporal alignment module determines the temporal position of neuron activation based on the EEG signals, and then uses the improved classical GLM method to process the fNIRS signals to perform temporal alignment on the preprocessed EEG signals and fNIRS signals;

[0101] The causal analysis module respectively uses empirical mode decomposition (EMD) on the EEG signals and fNIRS signals to obtain their respective main factor modal information; based on the main factor modal information of the EEG signals and the main factor modal information of the fNIRS signals, convergence cross mapping is used to solve for the causal relationship between the EEG signals and the fNIRS signals.

[0102] The above embodiments are only illustrative of the principles and effects of the present invention, and are not used to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS, characterized in that Including the following steps: Step 1, create a psychological experiment paradigm for guiding patients to conduct cognitive ability tests; Step 2, collect two-modal data of asynchronous EEG signals and fNIRS signals generated by patients during the execution of the psychological experiment paradigm, and label the collected EEG signals and fNIRS signals; Step 3, signal preprocessing, respectively perform channel selection, filtering, epochs segmentation, downsampling, and hemoglobin concentration conversion processing on the collected EEG signals and fNIRS signals; Step 4, by calculating the oxyhemoglobin concentration data in the fNIRS signals, adopt a method of screening EEG signal spatial channels based on the activation regions determined by the hemoglobin concentration distribution to spatially align the preprocessed EEG signals and fNIRS signals; determine the temporal position of neuron activation based on the EEG signals, and then use the improved classical GLM method to process the fNIRS signals. The preprocessed EEG signals and fNIRS signals are temporally aligned according to the following steps: The spatially aligned EEG signals are segmented into multiple EEG time-series segments of time windows by using overlapping sliding windows And each time series segment contains an activation time series, where k ∈ K, and K represents the number of divisions of single-trial experiments; Extract the average power values of the EEG signals in the four frequency bands of Delta, Theta, Alpha, and Beta under each time window to obtain the multi-band power value change state in time series; Based on the multi-band power value change state, establish a linear predictor, and combine it with the fNIRS signals to establish a generalized functional linear model as follows: μ = E[Y] = g -1 (η); where Y i is the fNIRS time series after time alignment, η is the linear predictor, μ and E[Y] are the expected values of the response variable Y i , Θ i is the natural parameter related to σ, b(Θ i ) is the cumulant generating function specific to the fNIRS distribution, c(Y i , φ) is the normalization term specific to the fNIRS distribution to ensure that the integral value of the probability density function is 1, φ is the dispersion parameter of the model, X1,..., X n are all polynomials composed of the change states of multi-band power values in time series, serving as the predictor variables in the model, β0 is the constant term of the model, β1, β2,... β n are all vectors of regression coefficients to be estimated, representing the influence of each predictor variable on the response variable, n is the number of samples of the EEG signal, g is the link function of the generalized function linear model, and L(β) is the likelihood function used to determine the parameter values; Use the generalized functional linear model to re-model the hemodynamic response on the time series of EEG signal activation to complete the temporal alignment of the EEG signals and fNIRS signals; Step 5, after the EEG signals and fNIRS signals are spatially and temporally aligned, respectively perform empirical mode decomposition on the EEG signals and fNIRS signals to obtain their respective main factor modal information; based on the main factor modal information of the EEG signals and the main factor modal information of the fNIRS signals, use convergent cross mapping to solve to obtain the causal relationship between the EEG signals and the fNIRS signals.

2. The causal analysis method for brain imaging signals oriented to asynchronous EEG-fNIRS according to claim 1, characterized in that In the said Step 4, the process of spatially aligning the preprocessed EEG signals and fNIRS signals includes the steps: Process the fNIRS signal using the modified Lambert-Beer law to obtain the concentration of oxyhemoglobin in multiple channels where k ∈ K, representing the number of divisions of single experimental trials; Then extract the active coefficients of all channels, that is, the brain region active information, through the improved common spatial pattern algorithm; select the spatial multi-dimensional channels that make up the maximum active information to obtain the active degree distribution of different brain regions. The calculation formula is as follows: y k = max([M i N j ); where, S i represents different activity levels, S m represents the source signal common to different channel activity levels, C i represents the common spatial pattern of S i , C m represents the common spatial pattern of S m , y k represents the maximum active information of the spatial multi-dimensional channels, [M i N j is the subspace combination of the spatial channels; Based on the brain region active information provided by the fNIRS signals and the active degree distribution of different brain regions, screen the EEG signal channels of different brain regions to achieve the spatial alignment of the EEG signals and fNIRS signals.

3. A method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS according to claim 2, characterized in that, The process of screening the EEG signal channels of different brain regions based on the brain region active information provided by the fNIRS signals and the active degree distribution of different brain regions includes: Using the Brodmann area system, subdivide the brain regions into 52 refined partitions. On this basis, correspond the positions of each channel of the fNIRS signals to the Brodmann areas one by one, and identify the partition numbers corresponding to each active channel region; Within the scope of the active brain regions, all EEG signal channels and fNIRS signal channels adjacent to the specified partition serial number are screened out, and finally, the alignment of EEG signals and fNIRS signals at the spatial level is achieved.

4. A method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS according to claim 3, characterized in that After the spatial alignment and temporal alignment of EEG signals and fNIRS signals are completed in step 5, the process of obtaining the respective main factor modal information for EEG signals and fNIRS signals by using empirical mode decomposition (EMD) respectively includes: For EEG signals: A sliding time window is constructed to gradually read the temporal potential values of EEG signals to find all their local maximum and minimum values; then, the cubic spline interpolation method is used to construct the upper envelope and the lower envelope for the maximum and minimum values respectively, and the mean value of the envelope is removed from the EEG signal to obtain the intrinsic mode function components, that is, the main factor modal information. The process of obtaining the main factor modal information for fNIRS signals by using empirical mode decomposition is the same as the process of obtaining the main factor modal information for EEG signals by using empirical mode decomposition.

5. A causal analysis method for brain imaging signals for asynchronous EEG-fNIRS according to claim 4, characterized in that The process of obtaining the causal relationship between EEG signals and fNIRS signals by using convergent cross-mapping based on the main factor modal information of EEG signals and the main factor modal information of fNIRS signals includes: For the intrinsic mode function components decomposed from EEG signals and fNIRS signals, convergent cross-mapping is performed component by component, and weighted summation is carried out to obtain the Pearson coefficient representing the causal relationship between EEG signals and fNIRS signals, and thus the causal relationship between EEG signals and fNIRS signals can be obtained; its calculation formula is as follows: L i = (X E (t i ), X E (t i + τ), …, X E (t i + (m - 1)τ)); Z pre Z(t) = a0 + a1L i + a2L i-1 + … + a j LL i-j ; Among them, L i is the result after the state embedding of the EEG signal X E , m is the embedding dimension, τ is the delay time, i is the embedding index, Z pre is the regression result of predicting the fNIRS signal Z f , a j is the regression coefficient, j is the number of delays used in the regression, L i-1 , …, L i-j are the historical state vectors of the first to j time points of the index used in the regression, Cov(Z pre , Z f ) is the covariance, σZ pre is the standard deviation of all data values in the regression result Z pre , σZ f is the standard deviation of all data values in the fNIRS signal Z f .

6. An apparatus for causal analysis of brain imaging signals for asynchronous EEG-fNIRS, characterized in that, Including: An experimental paradigm module, a data acquisition module, a data preprocessing module, a temporal alignment module, and a causal analysis module; through the above modules, a method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS as described in any one of claims 1-5 is implemented.

Images