Asynchronous EEG-fNIRS-oriented brain image signal causal analysis method and device

Abstract

The invention belongs to the field of electroencephalogram signal analysis, and particularly relates to an asynchronous EEG-fNIRS-oriented brain image signal causal analysis method and device, and the method comprises the steps: creating a psychological experiment normal form for guiding a patient to carry out a cognitive ability test; collecting and preprocessing asynchronous EEG and fNIRS signals generated by the patient in the process of executing the psychological experiment normal form; extracting spatial features from the fNIRS signals to process EEG signals, so that the two signals are spatially aligned; time sequence features are extracted from the EEG signals to process the fNIRS signals, so that the time sequence features and the fNIRS signals are aligned; and analyzing the causal relationship between the EEG and the fNIRS signal after space and time alignment. According to the method, the spatial-temporal information of various modal signals collected by asynchronous non-joint equipment is fully utilized, the flexibility and accuracy of the method can support deep analysis of brain region states, and an effective tool is provided for neurovascular coupling analysis and clinical application of EEG and fNIRS.

Description

Technical Field

[0001] The present invention belongs to the field of electroencephalogram signal analysis, and specifically relates to a brain image signal causal analysis method and device for asynchronous EEG-fNIRS. Background Art

[0002] Neurovascular coupling (NVC) refers to the relationship between neuronal activation and blood flow and metabolism in the corresponding area. This mechanism is jointly participated by neurons, glial cells and vascular cells, revealing the close connection between neural electrical activity and dynamic changes in blood flow. Under external stimulation, the activation of neurons not only produces neural electrical activity, but also promotes blood flow to active brain areas to meet their increased metabolic needs. This process is reflected in the changes in blood oxygen levels, specifically the increase in the concentration of oxyhemoglobin (HbO2) and the decrease in the concentration of deoxyhemoglobin (HbR). In recent years, NVC analysis has become a research hotspot in the field of brain science, providing a theoretical basis for revealing the relationship between brain neural activity and hemodynamics. The use of multimodal brain imaging technology is one of the important directions in the field of NVC research. The analysis of the causal relationship of multimodal brain imaging signals helps to explain the deep state information of the human brain system and the source of the disease. Therefore, it is crucial to develop a systematic and reliable method for causal analysis of brain imaging signals.

[0003] In the study of causal analysis methods of multimodal brain imaging signals, electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) are the two most commonly used neuroimaging techniques. EEG can provide information with high temporal resolution by measuring changes in potential on the surface of the scalp, which is suitable for capturing rapid neural electrical activity. However, the spatial resolution of EEG is limited and cannot capture data deep in the cortex. fNIRS uses near infrared light of different wavelengths of 650-950nm to measure changes in HbO2 and HbR concentrations in brain tissue. Although the temporal resolution is limited, it can provide high spatial resolution. EEG and fNIRS each have their own unique advantages and are complementary in temporal and spatial characteristics, making them powerful tools for studying NVC mechanisms and able to deeply explore brain neural activity under different cognitive tasks. However, most existing traditional neurovascular coupling studies are not effective in describing the causal relationship of EEG-fNIRS signals and are limited in the utilization of spatiotemporal signals in brain regions.

[0004] For example, the Chinese patent application with publication number CN118626786A discloses a causal analysis method for spatiotemporal multi-scale EEG and functional near-infrared signals. The method obtains the causal relationship of multimodal signals based on EEG-fNIRS signals at different time scales, integrates the causal relationship through the weighted summation method, and obtains the causal analysis results.

[0005] For example, the Chinese patent application with publication number CN117474103A discloses a cross-modal causal relationship analysis method based on EEG-fNIRS. This method uses variational modal decomposition and short-time Fourier transform to process multimodal signals, and completes neurovascular coupling analysis on different characteristic frequency bands based on causal strength values.

[0006] These previous inventions are mainly aimed at EEG-fNIRS signals under synchronous acquisition. On the one hand, due to the limitation of using EEG-fNIRS combined acquisition equipment, the temporal and spatial resolution of the two modal signals are relatively insufficient; on the other hand, for asynchronous EEG-fNIRS signals, the temporal and spatial information of the two modal signals are not fully utilized, and there is a lack of a standardized and reliable framework, making it difficult to use in clinical applications. Summary of the invention

[0007] In view of this, the present invention proposes a causal analysis method for asynchronous EEG-fNIRS brain imaging signals. This method extracts and utilizes the spatiotemporal information of the two modalities of asynchronous EEG-fNIRS signals through spatiotemporal alignment technology to analyze the spatiotemporal causal relationship of the aligned data. This method not only gets rid of the dependence of traditional multimodal data analysis on equipment, but also improves the flexibility and accuracy of causal analysis of brain imaging signals.

[0008] The technical solution of the present invention is as follows:

[0009] A causal analysis method for asynchronous EEG-fNIRS brain imaging signals comprises the following steps:

[0010] Step 1: Create a psychological experimental paradigm to guide patients to take cognitive ability tests;

[0011] Step 2: Collect two modal data, namely, asynchronous EEG signals and fNIRS signals generated by the patient in the psychological experimental paradigm, and assign labels to the collected EEG signals and fNIRS signals;

[0012] Step 3: Signal preprocessing: channel selection, filtering, epochs segmentation, downsampling, and hemoglobin concentration conversion are performed on the collected EEG signals and fNIRS signals respectively;

[0013] Step 4: By calculating the oxygenated hemoglobin concentration data in the fNIRS signal, the method of screening the spatial channels of the EEG signal based on the activation area determined by the hemoglobin concentration distribution is used to spatially align the preprocessed EEG signal and the fNIRS signal; the temporal position of neuronal activation is determined based on the EEG signal, and then the fNIRS signal is processed using the improved classical GLM method to temporally align the preprocessed EEG signal and the fNIRS signal;

[0014] Step 5. After the EEG signal and fNIRS signal are spatially aligned and temporally aligned, the EEG signal and fNIRS signal are subjected to empirical mode decomposition (EMD) to obtain their respective main factor modal information. Based on the main factor modal information of the EEG signal and the main factor modal information of the fNIRS signal, a convergent cross-mapping solution is used to obtain the causal relationship between the EEG signal and the fNIRS signal.

[0015] Furthermore, in step 4, the process of spatially aligning the preprocessed EEG signal and the fNIRS signal comprises the steps of:

[0016] The fNIRS signal was processed using the modified Lambert-Beer law to obtain the multi-channel oxygenated hemoglobin concentration. ,in , represents the number of times a single experimental trial is divided; then the activity coefficients of all channels, that is, the activity information of the brain area, are extracted by improving the common spatial pattern algorithm; the spatial multidimensional channels that constitute the maximum activity information are selected to obtain the activity distribution of different brain areas; the calculation formula is as follows:

[0017] ;

[0018] ;

[0019] in, Represents different levels of activity. Represents the source signal common to the activity levels of different channels, represent The common space mode, represent The common space mode, Represents the maximum active information of the multi-dimensional channel of space, is the subspace combination of the spatial channel;

[0020] Based on the brain area activity information provided by the fNIRS signal and the activity distribution of different brain areas, the EEG signal channels of different brain areas are screened to achieve spatial alignment of the EEG signal and the fNIRS signal.

[0021] Furthermore, the process of screening EEG signal channels of different brain regions based on the brain region activity information and the activity distribution of different brain regions provided by the fNIRS signal includes:

[0022] The Brodmann partition system was used to subdivide the brain into 52 refined partitions. Based on this, the positions of each channel of the fNIRS signal were matched one by one with the Brodmann partitions, and the partition number corresponding to each active channel area was identified.

[0023] Within the range of active brain areas, all EEG signal channels and fNIRS signal channels adjacent to the specified partition number are screened out, and finally the EEG signal and fNIRS signal are aligned at the spatial level.

[0024] Furthermore, in step 4, the process of time-aligning the pre-processed EEG signal and the fNIRS signal comprises the following steps:

[0025] The spatially aligned EEG signal is divided into multiple EEG time series segments using overlapping sliding windows. , and each time slice contains an activation time sequence, where , represents the number of divisions of a single experimental trial;

[0026] Extract the average power value of the EEG signal in the four frequency bands of Delta, Theta, Alpha and Beta in each time window to obtain the change state of the multi-band power value in time series;

[0027] A linear predictor was established based on the change state of multi-band power values, and a generalized functional linear model was established in combination with fNIRS signals, as described below:

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] in, is the fNIRS time series after time alignment, is the linear predictor, and is the response variable The expected value of is with The relevant natural parameters, is the cumulant generating function specific to the fNIRS distribution, is a normalization term specific to the fNIRS distribution and is used to ensure that the integral of the probability density function is 1. is the dispersion parameter of the model, are all polynomials composed of the change states of multi-band power values ​​in time series, which are used as predictive variables in the model. is the constant term of the model, are all regression coefficient vectors to be estimated, indicating the impact of each predictor variable on the response variable. is the number of samples of EEG signal, is the link function of the generalized functional linear model, is the likelihood function used to determine the parameter value;

[0033] The generalized linear norm function was used to remodel the cerebral hemodynamic response based on the timing of EEG signal activation, thus completing the time alignment of EEG and fNIRS signals.

[0034] Furthermore, in step 5, after the EEG signal and the fNIRS signal are spatially aligned and temporally aligned, the process of respectively applying empirical mode decomposition (EMD) to the EEG signal and the fNIRS signal to obtain their respective main factor modal information includes:

[0035] For EEG signals: construct a sliding time window to gradually read the time series potential values ​​of the EEG signal to find all its local maxima and minima; then use the spline interpolation method to construct the upper envelope and lower envelope for the maximum and minimum values ​​respectively, and remove the mean of the envelope from the EEG signal to obtain the intrinsic mode function component, that is, the main factor modal information;

[0036] The process of obtaining the main factor modal information based on empirical mode decomposition of the fNIRS signal is the same as the process of obtaining the main factor modal information based on empirical mode decomposition of the EEG signal.

[0037] Furthermore,

[0038] According to the main factor modal information of the EEG signal and the main factor modal information of the fNIRS signal, the process of using convergent cross mapping to solve and obtain the causal relationship between the EEG signal and the fNIRS signal includes:

[0039] The intrinsic mode function components decomposed from the EEG signal and the fNIRS signal are converged and cross-mapped component by component, and the weighted sum is used to obtain the Pearson coefficient representing the causal relationship between the EEG signal and the fNIRS signal, so that the causal relationship between the EEG signal and the fNIRS signal can be obtained; the calculation formula is as follows:

[0040] ;

[0041] ;

[0042] ;

[0043] in, For EEG signals The result after state embedding, m is the embedding dimension, is the delay time, i is the embedded index, To predict fNIRS signals The regression results of are regression coefficients, j is the number of lags used in the regression, is the historical state vector of the index 1 to j time points before used in regression, is the covariance, The regression results The standard deviation of all data values ​​in , fNIRS signal The standard deviation of all data values ​​in .

[0044] A brain image signal causal analysis device for asynchronous EEG-fNIRS includes: an experimental paradigm module, a data acquisition module, a data preprocessing module, a time alignment module and a causal analysis module; the above-mentioned brain image signal causal analysis method for asynchronous EEG-fNIRS is implemented through the above-mentioned modules.

[0045] The present invention provides a brain imaging signal causal analysis method and device for asynchronous EEG-fNIRS. For EEG signals and fNIRS signals with time and space differences under asynchronous acquisition, the spatial resolution of fNIRS signals is higher than that of EEG signals, and spatial features are extracted from fNIRS signals to process EEG signals to align the channel areas of the two. By utilizing the millisecond-level temporal resolution characteristics of EEG signals, transient EEG oscillation characteristics can be captured, and timing features can be extracted from EEG signals and used to process fNIRS signals to align the timing of the two, and then the main factor modal information of each can be obtained by using empirical mode decomposition (EMD); based on the main factor modal information of EEG signals and the main factor modal information of fNIRS signals, convergent cross-mapping is used to solve and obtain the causal relationship between EEG signals and fNIRS signals, thereby realizing the causal relationship analysis of multi-modal brain imaging data under asynchronous acquisition. The present invention fully utilizes the spatiotemporal information of multiple modal signals collected by asynchronous non-joint devices. Its flexibility and accuracy can support in-depth analysis of brain region states and provide an effective tool for neurovascular coupling analysis and clinical application of EEG and fNIRS. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1This is a flow chart of a method for causal analysis of brain imaging signals for asynchronous EEG-fNIRS according to an embodiment;

[0047] Figure 2 N-back paradigm diagram used in the embodiment;

[0048] Figure 3 Technical roadmap for causal relationship analysis of EEG signals and fNIRS signals

[0049] Figure 4 This is a graph of EEG and fNIRS time-aligned for a single trial in the example. DETAILED DESCRIPTION

[0050] The present invention is described in detail below with reference to the accompanying drawings and embodiments, but the embodiments of the present invention are not limited thereto.

[0051] This embodiment provides a causal analysis method for asynchronous EEG-fNIRS brain imaging signals, such as Figure 1 As shown, the following steps are included:

[0052] Step 1. Create an experimental paradigm:

[0053] The experimental paradigm module is used to present the guided paradigm to the subjects, so as to guide the subjects to activate the brain areas related to cognitive functions through task behaviors. This embodiment adopts the psychological experimental paradigm N-back for working memory ability test, which is produced by E-PRIME software, such as Figure 2 As shown. This paradigm is based on the classic psychological task N-back experiment, which tests the patient's attention and memory level. The experimental paradigm is divided into three groups according to difficulty: 0-back, 1-back, and 2-back. Participants sit in front of a computer and observe the sequence of numbers or letters presented on the screen. In each task, each letter is presented for 0.5 seconds, followed by a blank screen for 2 seconds until the next letter appears. Participants need to judge as quickly and accurately as possible whether the number or letter currently presented is the same as the number or letter n positions ago. If the same, press the "yes" key on the keyboard; if different, press the "no" key on the keyboard. The duration of a single experimental trial is 2 seconds, including a prompt time of 0.5 seconds and a rest time of 1.5 seconds. The E-PRIME experimental paradigm consists of three rounds, each round has 3 blocks, and each block contains 15 experimental trials.

[0054] Step 2: Signal acquisition:

[0055] The data acquisition module is used to collect the asynchronous EEG signal and fNIRS signal data generated by the subjects during the execution of the psychological experimental paradigm, and transmit the signal data to the corresponding data acquisition software, mark the signal data, and store it in the system.

[0056] This embodiment uses a near-infrared brain imaging device with fNIRS signal acquisition function for the frontal, parietal and temporal regions to collect fNIRS signals. The device includes a multi-channel head-mounted device and a terminal with fNIRS signal acquisition software installed. EEG signals are collected using the eegoTM mylab 32-lead EEG acquisition device, which mainly consists of a 32-channel EEG cap, a 32-channel 16kHz amplifier, and a terminal notebook equipped with eego64 EEG signal acquisition software. The acquisition and processing software can present data in real time and perform signal processing, and also has an event marking function. The head-mounted device is connected to the notebook wirelessly to transmit the collected EEG signals and fNIRS signals to the terminal notebook in real time.

[0057] Step 3: Data preprocessing:

[0058] Read the signal data of the two modalities of EEG signal and fNIRS signal stored in the system in step 2, and perform preprocessing including channel selection, filtering, downsampling, epochs segmentation, hemoglobin concentration conversion, etc., and store the preprocessed EEG signal and fNIRS signal in the system. Among them:

[0059] The filtering operation is used to filter out noise interference such as motion artifacts, eye movements, and heartbeats in the signal, while retaining the EEG signal and fNIRS signal in the target frequency band. Specifically, the EEG signal data is processed using a sixth-order Butterworth bandpass filter (bidirectional zero-phase filter) with a passband frequency range of 1 Hz to 30 Hz; the fNIRS signal is filtered through a discrete cosine transform filter with an effective frequency band of 0.001 Hz to 0.5 Hz.

[0060] The downsampling operation is used to reduce the length of the collected EEG signal and fNIRS signal data.

[0061] In the epochs segmentation stage, the EEG signal and fNIRS signal data are divided into independent trials with different experimental times according to the marking points in the acquisition process of step 2. For fNIRS signals, the data of 0–30 seconds after the task block marking point is intercepted as a single experimental segment; for EEG signals, the data of 0–2 seconds after each trial marking point is intercepted to form a single trial EEG segment.

[0062] The hemoglobin concentration conversion operation is used to convert the infrared light intensity changes continuously collected by the fNIRS device into the relative concentration changes of oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR). The attenuation changes of light intensity at different wavelengths are calculated by using the modified Lambert-Beer law, and the dynamic change value of hemoglobin concentration is finally solved by combining the optical path factor (DPF) and the differential absorption coefficient. The calculation formula is as follows:

[0063] ;

[0064] Where d is the distance between the light source and the detector; and At the corresponding wavelength The concentration of oxygenated hemoglobin and the extinction coefficient of deoxygenated hemoglobin; is the wavelength after sampling time The change of optical density at The wavelength The differential path length factor (i=0,1); and are the relative changes of oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration, respectively; is the time interval between two adjacent samplings.

[0065] It should be noted that the hemoglobin concentration conversion method of this embodiment is applicable to various hemoglobin data types such as oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration. In order to more clearly explain the technical solution of this embodiment, the fNIRS data involved in the subsequent description specifically refers to the oxygenated hemoglobin (HbO) concentration data obtained based on the fNIRS technology.

[0066] Step 4: Spatiotemporal alignment and causal analysis:

[0067] The preprocessed EEG signal and fNIRS signal are spatially aligned and temporally aligned, and then the aligned EEG signal and fNIRS signal are subjected to causal relationship analysis. The route of spatially aligning, temporally aligning, and causal relationship analysis of the EEG signal and fNIRS signal in this embodiment is as follows: Figure 3 As shown:

[0068] This implementation uses the activation area obtained by the hemoglobin concentration distribution to screen the spatial channels of the EEG signal to align the two modal data; the spatial alignment method aims to extract features from the fNIRS signal with a high level of spatial resolution and use it to process the EEG signal to align the channel areas of the two. This method is based on the activation area determined by the hemoglobin concentration distribution, and screens the spatial channels of the EEG signal to achieve spatial alignment of the two modal data. The specific steps include:

[0069] The fNIRS signal was processed using the modified Lambert-Beer law to obtain the multi-channel oxygenated hemoglobin concentration. ,in , represents the number of times a single experimental trial is divided; then the activity coefficients of all channels, that is, the activity information of the brain area, are extracted through the improved common spatial pattern (CSP) algorithm; and the spatial multidimensional channels that constitute the maximum activity information are selected to obtain the activity distribution of different brain areas. The calculation formulas are shown in formula (2) and formula (3):

[0070] ;

[0071] ;

[0072] in, Represents different levels of activity. Represents the source signal common to the activity levels of different channels, represent The common space mode, represent The common space mode, Represents the maximum active information of the multi-dimensional channel of space, is the subspace combination of spatial channels.

[0073] Through the distribution information of the activity level of different brain regions, the channels of the EEG signal are multi-layered. Since the EEG signal follows the international 10-20 channel standard, the channel position is fixed; while the number of channels of the fNIRS signal is denser, and the channel position is relatively flexible. In order to achieve multi-layer screening of EEG signal channels, this embodiment uses the brain area activity information provided by the fNIRS signal, combined with the distribution of activity levels of different brain regions, to screen the EEG signal channels of different brain regions. Specifically: Using the Brodmann partition system, the brain region is subdivided into 52 refined partitions. On this basis, the channel positions of the fNIRS signal are matched one by one with the Brodmann partitions, and the partition number corresponding to each active channel area is identified. Within the range of active brain areas, all EEG signal channels and fNIRS signal channels adjacent to the specified partition number are screened, and finally the precise alignment of the EEG signal and the fNIRS signal at the spatial level is achieved, giving full play to the advantages of the high spatial resolution of fNIRS.

[0074] The time alignment method aims to extract features from high-temporal-resolution EEG signals and use them to process fNIRS signals to align the two in time. This method is based on the temporal position of neuronal activation obtained from EEG signals, and then uses the improved classical GLM method to process fNIRS signals to synchronize the time series of the two modal data. The specific steps include:

[0075] Set a sliding time window with a duration of 1000ms and a window overlap rate of 50% to split the spatially aligned EEG signal into EEG time series segments of multiple time windows. ,in , Indicates the number of times a single experimental trial is divided. Subsequently, the average power value of the EEG signal in the four frequency bands of Delta (0.5-4hz), Theta (4-8hz), Alpha (8-13hz), and Beta (13-30Hz) in each time window is extracted to obtain the change state of the multi-band power value in the time series. , its calculation formula is shown in formula (4):

[0076] (4);

[0077] in, express The complex representation of the ith sub-segment in the frequency domain f is obtained by Fourier transform. express The model, To sum the moduli of all sub-segments, To further sum the modulus values ​​corresponding to all frequencies;

[0078] A linear predictor is established based on the change state of multi-band power values, and a generalized function linear model is established in combination with the fNIRS signal, as shown in formulas (5), (6), (7), and (8). The generalized function linear model is used to remodel the cerebral hemodynamic response based on the timing of EEG signal activation, and the time alignment of the EEG signal and the fNIRS signal is completed. The single-trial time-aligned EEG signal and fNIRS signal are shown in Figure 4 As shown:

[0079] (5);

[0080] (6);

[0081] (7);

[0082] (8);

[0083] in, is the fNIRS time series after time alignment, is the linear predictor, and is the response variable The expected value of is with The relevant natural parameters, is the cumulant generating function specific to the fNIRS distribution, is a normalization term specific to the fNIRS distribution and is used to ensure that the integral of the probability density function is 1. is the dispersion parameter of the model, are all polynomials composed of the change states of multi-band power values ​​in time series, which are used as predictive variables in the model. is the constant term of the model, are all regression coefficient vectors to be estimated, indicating the impact of each predictor variable on the response variable. is the number of samples of EEG signal, is the link function of the generalized functional linear model, is the likelihood function used to determine the parameter value;

[0084] The causal analysis method is used to perform causal relationship analysis on the EEG signal and fNIRS signal after spatiotemporal alignment. This method is based on the empirical mode decomposition (EMD) to obtain the main factor modal information, and uses convergent cross-mapping to solve the causal relationship between the two modal data. The specific steps include:

[0085] For the EEG signals and fNIRS signals that have completed spatial alignment and temporal alignment, empirical mode decomposition is performed respectively. That is, a sliding time window is constructed for the EEG signals and fNIRS signals to gradually read the local maximum and minimum values ​​found, and the upper and lower envelopes are constructed for the maximum and minimum values ​​respectively through the spline interpolation method. The mean of the envelopes is removed from the EEG signals and fNIRS signals to obtain the intrinsic mode function components, that is, the main factor modal information is shown in the formula:

[0086] (9);

[0087] (10) ;

[0088] (11);

[0089] in, represents the upper envelope coordinates, represents the coordinates of the upper and lower envelopes, m is the mean value of the envelope, X is the two modal signals, is the intrinsic mode function component obtained by continuously decomposing X, r is the remaining component after removing the intrinsic mode function from X in each decomposition, and t represents the time step.

[0090] For the intrinsic mode function components decomposed from the two signals, convergent cross-mapping is performed component by component, and the weighted sum is used to obtain the Pearson coefficient representing the causal relationship between the two modal signals of EEG signal and fNIRS signal, thereby obtaining the causal relationship between EEG signal and fNIRS signal; its calculation formula is as shown in formula (12)-formula (14):

[0091] (12);

[0092] (13);

[0093] (14);

[0094] in, For EEG signals The result after state embedding, m is the embedding dimension, is the delay time, i is the embedded index, To predict fNIRS signals The regression results of are regression coefficients, j is the number of lags used in the regression, is the historical state vector of the index 1 to j time points before used in regression, is the covariance, The regression results The standard deviation of all data values ​​in , fNIRS signal The standard deviation of all data values ​​in .

[0095] Based on the above method, this embodiment also provides a brain image signal causal analysis device for asynchronous EEG-fNIRS, which includes: an experimental paradigm module, a data acquisition module, a data preprocessing module, a time alignment module and a causal analysis module connected in sequence; wherein:

[0096] The experimental paradigm module is used to present a guiding paradigm to patients, instructing them to activate brain areas related to cognitive functions through task behaviors. The paradigm used is the N-back psychological experimental paradigm for cognitive ability testing. By setting up experimental combinations of multiple difficulties, the patient's attention and memory levels are tested to activate the patient's working memory brain area. Different levels of difficulty will stimulate different levels of mental workload in patients.

[0097] The data acquisition module is used to collect the asynchronous EEG signals and fNIRS signals generated by the subjects when executing the experimental paradigm, and to assign labels to the EEG signals;

[0098] The data preprocessing module is used to receive the collected EEG signals and fNIRS signals, and perform filtering, epochs segmentation, downsampling and hemoglobin concentration conversion on the EEG signals and fNIRS signals respectively;

[0099] The spatial alignment module calculates the oxygenated hemoglobin concentration data in the fNIRS signal and uses the method of screening the spatial channels of the EEG signal based on the activation area determined by the hemoglobin concentration distribution to spatially align the preprocessed EEG signal and the fNIRS signal.

[0100] The time alignment module determines the temporal position of neuronal activation based on the EEG signal, and then uses the improved classical GLM method to process the fNIRS signal to time align the preprocessed EEG signal and fNIRS signal;

[0101] The causal analysis module uses empirical mode decomposition (EMD) for EEG signals and fNIRS signals to obtain their respective main factor modal information; based on the main factor modal information of EEG signals and the main factor modal information of fNIRS signals, convergent cross mapping is used to solve and obtain the causal relationship between EEG signals and fNIRS signals.

[0102] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A causal analysis method for asynchronous EEG-fNIRS brain imaging signals, characterized in that: The following steps are involved: Step 1: Create a psychological experimental paradigm to guide patients to take cognitive ability tests; Step 2: Collect two modal data, namely, asynchronous EEG signals and fNIRS signals generated by the patient in the psychological experimental paradigm, and assign labels to the collected EEG signals and fNIRS signals; Step 3: Signal preprocessing: channel selection, filtering, epochs segmentation, downsampling, and hemoglobin concentration conversion are performed on the collected EEG signals and fNIRS signals respectively; Step 4: By calculating the oxygenated hemoglobin concentration data in the fNIRS signal, the method of screening the spatial channels of the EEG signal based on the activation area determined by the hemoglobin concentration distribution is used to spatially align the preprocessed EEG signal and the fNIRS signal; the temporal position of neuronal activation is determined based on the EEG signal, and then the fNIRS signal is processed using the improved classical GLM method to temporally align the preprocessed EEG signal and the fNIRS signal; Step 5. After the EEG signal and fNIRS signal are spatially aligned and temporally aligned, empirical mode decomposition is used on the EEG signal and fNIRS signal to obtain their respective main factor modal information. Based on the main factor modal information of the EEG signal and the main factor modal information of the fNIRS signal, convergent cross mapping is used to solve and obtain the causal relationship between the EEG signal and the fNIRS signal.

2. A method for causal analysis of brain image signals based on asynchronous EEG-fNIRS according to claim 1, characterized in that: In step 4, the process of spatially aligning the preprocessed EEG signal and the fNIRS signal includes the following steps: The fNIRS signal was processed using the modified Lambert-Beer law to obtain the multi-channel oxygenated hemoglobin concentration. ,in , represents the number of divisions of a single experimental trial; Then, the activity coefficients of all channels, i.e., the activity information of brain regions, are extracted by improving the common spatial pattern algorithm; the spatial multidimensional channels that constitute the maximum activity information are selected to obtain the activity distribution of different brain regions; the calculation formula is as follows: ； ;in, Represents different levels of activity. Represents the source signal common to the activity levels of different channels, represent The common space mode, represent The common space mode, Represents the maximum active information of the multi-dimensional channel of space, is the subspace combination of spatial channels; Based on the brain area activity information provided by the fNIRS signal and the activity distribution of different brain areas, the EEG signal channels of different brain areas are screened to achieve spatial alignment of the EEG signal and the fNIRS signal.

3. The method for causal analysis of brain image signals based on asynchronous EEG-fNIRS according to claim 2, characterized in that: The process of screening EEG signal channels of different brain regions based on the brain region activity information provided by the fNIRS signal and the activity distribution of different brain regions includes: The Brodmann partition system was used to subdivide the brain into 52 refined partitions. Based on this, the positions of each channel of the fNIRS signal were matched one by one with the Brodmann partitions, and the partition number corresponding to each active channel area was identified. Within the range of active brain areas, all EEG signal channels and fNIRS signal channels adjacent to the specified partition number are screened out, and finally the EEG signal and fNIRS signal are aligned at the spatial level.

4. The method for causal analysis of brain image signals based on asynchronous EEG-fNIRS according to claim 2, characterized in that: In step 4, the process of time-aligning the pre-processed EEG signal and the fNIRS signal includes the following steps: The spatially aligned EEG signal is divided into multiple EEG time series segments using overlapping sliding windows. , and each time slice contains an activation time sequence, where , represents the number of divisions of a single experimental trial; Extract the average power value of the EEG signal in the four frequency bands of Delta, Theta, Alpha and Beta in each time window to obtain the change state of the multi-band power value in time series; A linear predictor was established based on the change state of multi-band power values, and a generalized functional linear model was established in combination with fNIRS signals, as described below: ； ； ； ； in, is the fNIRS time series after time alignment, is the linear predictor, and is the response variable The expected value of is with The relevant natural parameters, is the cumulant generating function specific to the fNIRS distribution, is a normalization term specific to the fNIRS distribution and is used to ensure that the integral of the probability density function is 1. is the dispersion parameter of the model, are all polynomials composed of the change states of multi-band power values ​​in time series, which are used as predictive variables in the model. is the constant term of the model, are all regression coefficient vectors to be estimated, indicating the impact of each predictor variable on the response variable. is the number of samples of EEG signal, is the link function of the generalized functional linear model, is the likelihood function used to determine the parameter value; The generalized linear norm function was used to remodel the cerebral hemodynamic response based on the timing of EEG signal activation, thus completing the time alignment of EEG and fNIRS signals.

5. The method for causal analysis of brain image signals based on asynchronous EEG-fNIRS according to claim 4, characterized in that: After the EEG signal and the fNIRS signal are spatially aligned and temporally aligned, the process of using empirical mode decomposition (EMD) to respectively perform the EEG signal and the fNIRS signal to obtain their respective main factor modal information includes: For EEG signals: construct a sliding time window to gradually read the time series potential values ​​of the EEG signal to find all its local maxima and minima; then use the spline interpolation method to construct the upper envelope and lower envelope for the maximum and minimum values ​​respectively, and remove the mean of the envelope from the EEG signal to obtain the intrinsic mode function component, that is, the main factor modal information; The process of obtaining the main factor modal information based on empirical mode decomposition of the fNIRS signal is the same as the process of obtaining the main factor modal information based on empirical mode decomposition of the EEG signal.

6. The method for causal analysis of brain image signals based on asynchronous EEG-fNIRS according to claim 5, characterized in that: According to the main factor modal information of the EEG signal and the main factor modal information of the fNIRS signal, the process of using convergent cross mapping to solve and obtain the causal relationship between the EEG signal and the fNIRS signal includes: The intrinsic mode function components decomposed from the EEG signal and the fNIRS signal are converged and cross-mapped component by component, and the weighted sum is used to obtain the Pearson coefficient representing the causal relationship between the EEG signal and the fNIRS signal, so that the causal relationship between the EEG signal and the fNIRS signal can be obtained; the calculation formula is as follows: ； ； ； in, For EEG signals The result after state embedding, m is the embedding dimension, is the delay time, i is the embedded index, To predict fNIRS signals The regression results of are regression coefficients, j is the number of lags used in the regression, is the historical state vector of the index 1 to j time points before used in regression, is the covariance, The regression results The standard deviation of all data values ​​in , fNIRS signal The standard deviation of all data values ​​in .

7. A brain image signal causal analysis device for asynchronous EEG-fNIRS, characterized in that: include: An experimental paradigm module, a data acquisition module, a data preprocessing module, a time alignment module and a causal analysis module; the above modules are used to implement a causal analysis method for asynchronous EEG-fNIRS brain imaging signals as described in any one of claims 1 to 6.

Images