EEG-fNIRS Electrode Alignment Method and System Based on Brain Region Division

Abstract

This invention discloses an EEG-fNIRS electrode arrangement method and system based on brain region segmentation, relating to the field of emotion recognition and classification. The method includes: simultaneously acquiring measured EEG signals and measured fNIRS signals of the brain during emotional cognitive activities using a 62-lead EEG recorder and an 18-lead near-infrared brain imaging system; dividing the brain into multiple regions and determining the electrode types and arrangement order in each region to obtain the EEG-fNIRS electrode arrangement for each region; determining the EEG-fNIRS correlation matrix features based on the measured EEG signals, measured fNIRS signals, and the electrode arrangement for each region; and performing emotion recognition based on the EEG-fNIRS correlation matrix features. In this invention, the segmentation of multiple brain regions and the introduction of fNIRS signals significantly improve the accuracy of emotion recognition.

Description

Technical Field

[0001] This invention relates to the field of EEG-fNIRS electrode arrangement used in emotion recognition and classification, and in particular to an EEG-fNIRS electrode arrangement method and system based on multi-brain region division. Background Technology

[0002] Emotions are often the result of coordinated activity of the cerebral cortex and subcortical nervous system triggered by external stimuli. Among numerous physiological signals, electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) signals are more widely used due to their low cost, high portability, and relatively simple testing. EEG signals record voltage fluctuations generated by brain activity on the scalp surface, directly reflecting the electrophysiological activity of neurons in the cerebral cortex and exhibiting high temporal resolution. fNIRS signals utilize the differences in the absorption rate of near-infrared light with wavelengths of 650nm-950nm in different areas of the brain to detect hemodynamic activity in the cerebral cortex, indirectly reflecting neuronal activity and its changes, and exhibiting high spatial resolution. Therefore, leveraging the complementary advantages of the temporal and spatial resolutions of EEG and fNIRS signals is of great significance for studying the mechanisms of brain neural activity in emotion recognition tasks.

[0003] Modern cognitive neuroscience indicates that there is a complex relationship between a person's emotional state and brain regions of the cerebral cortex, and that emotion is the result of the coordinated and interactive effects of multiple brain regions in the cerebral cortex. One of the key research focuses of EEG-fNIRS emotion recognition based on brain region segmentation is how to place and arrange EEG and fNIRS electrodes on the human head to better detect the brain regions' emotional response capabilities.

[0004] For synchronously acquired EEG-fNIRS signals, most existing studies employ horizontal or vertical scanning electrode arrangements. Recently, a nearest-neighbor EEG electrode arrangement method based on left and right brain regions was proposed in the paper Jang S, Moon SE, Lee J S. Convolutional neural network approach for EEG-based emotion recognition using brain connectivity and its spatial information [C]. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018:3066-3070. This method first selects the nearest EEG electrodes in the left brain, starting from the left frontal lobe, and then applies the same scheme to the right brain EEG electrodes. However, this electrode arrangement scheme only considers EEG signals and does not take into account the influence of multiple brain regions on the electrode arrangement. Therefore, this invention proposes an EEG-fNIRS electrode arrangement method and system based on brain region division. Summary of the Invention

[0005] The purpose of this invention is to provide an EEG-fNIRS electrode arrangement method and system based on brain region division. This method can divide the cerebral cortex into multiple regions and determine the arrangement of EEG electrodes and fNIRS electrodes in each divided brain region. Based on the measured EEG and fNIRS signals of each divided brain region and the electrode arrangement, an EEG-fNIRS correlation matrix feature is constructed and emotion recognition is performed. The division of multiple brain regions and the introduction of fNIRS signals greatly improve the accuracy of emotion recognition.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A method for EEG-fNIRS electrode arrangement based on brain region segmentation, the method comprising:

[0008] The measured EEG signals and measured fNIRS signals of the brain during emotional cognitive activities were simultaneously acquired using a 62-lead EEG recorder and an 18-lead near-infrared brain functional imaging system.

[0009] The brain region is divided into multiple regions, and the type and arrangement order of electrodes deployed in each region are determined to obtain the EEG-fNIRS electrode arrangement for each region. The electrode types deployed in each region include EEG electrodes and fNIRS electrodes. All EEG electrodes correspond one-to-one with the physical EEG electrodes deployed in the 62-lead EEG recorder. All fNIRS electrodes correspond one-to-one with the physical fNIRS electrodes deployed in the 18-lead near-infrared brain functional imaging system.

[0010] Based on the measured EEG signal and the measured fNIRS signal, as well as the EEG-fNIRS electrode arrangement of each brain region, the EEG-fNIRS correlation matrix characteristics of the EEG signal and fNIRS signal of each brain region are determined.

[0011] Sentiment recognition is performed based on the features of the EEG-fNIRS association matrix.

[0012] Optionally, the step of dividing the brain region into multiple regions and determining the type and arrangement order of electrodes in each divided brain region specifically includes:

[0013] The brain regions are divided into the frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe according to their functions.

[0014] Determine the type and arrangement order of electrodes deployed in each of the described brain regions.

[0015] Optionally, determining the type and arrangement order of electrodes deployed in each of the defined brain regions specifically includes:

[0016] The EEG electrodes are disposed in the frontal lobe region, the temporal lobe region, the central region, the parietal lobe region, and the occipital lobe region, and the fNIRS electrodes are disposed in the frontal lobe region and the temporal lobe region;

[0017] In the frontal lobe region and the temporal lobe region, the fNIRS electrode and the EEG electrode do not overlap;

[0018] The electrode arrangement order of the EEG electrodes and the fNIRS electrodes for each of the brain regions is determined.

[0019] Optionally, determining the electrode arrangement order of the EEG electrodes and fNIRS electrodes for each of the brain regions specifically includes:

[0020] The electrode arrangement order of each of the aforementioned brain regions is determined to obtain the brain region electrode arrangement order;

[0021] Based on the electrode arrangement order of the brain regions, the EEG electrode arrangement order and fNIRS electrode arrangement order within each of the divided brain regions are determined.

[0022] Optionally, determining the electrode arrangement order of each of the divided brain regions to obtain the brain region electrode arrangement order specifically includes:

[0023] The order of the EEG electrodes in the brain regions is determined by sequentially arranging the frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe regions.

[0024] The order of the brain regions of the fNIRS electrodes is determined by arranging them in the order of the frontal lobe region first and then the temporal lobe region.

[0025] Optionally, the order of EEG electrode arrangement and fNIRS electrode arrangement within each of the defined brain regions is determined, specifically including:

[0026] For each of the defined brain regions, a target electrode is randomly selected as the selection electrode; when the EEG electrode arrangement order is determined, the target electrode is the EEG electrode at the boundary of the defined brain region; when the fNIRS electrode arrangement order is determined, the target electrode is the fNIRS electrode at the boundary of the defined brain region.

[0027] Based on the principle of left-to-right selection, the electrode closest to the selected electrode is chosen as the next target electrode;

[0028] Using the next target electrode as the selected electrode, return to the step "select the electrode closest to the selected EEG electrode as the next target electrode based on the principle of left to right" until all target electrodes for the current brain region division are arranged.

[0029] The present invention also provides an EEG-fNIRS electrode arrangement system based on brain region division, the system comprising:

[0030] The signal acquisition module is used to simultaneously acquire measured EEG signals and measured fNIRS signals of the brain during emotional cognitive activities using a 62-lead EEG recorder and an 18-lead near-infrared brain functional imaging system.

[0031] The brain region division and electrode arrangement module is used to divide the brain into multiple regions and determine the electrode types and arrangement order in each divided brain region, thus obtaining the EEG-fNIRS electrode arrangement for each divided brain region. The electrode types deployed in each divided brain region include EEG electrodes and fNIRS electrodes. All EEG electrodes correspond one-to-one with the physical EEG electrodes deployed in the 62-lead EEG recorder. All fNIRS electrodes correspond one-to-one with the physical fNIRS electrodes deployed in the 18-lead near-infrared brain functional imaging system.

[0032] The correlation matrix feature extraction module is used to determine the EEG-fNIRS correlation matrix features of the EEG signals and fNIRS signals of each brain region based on the measured EEG signals, the measured fNIRS signals, and the EEG-fNIRS electrode arrangement of each brain region.

[0033] The emotion recognition module is used to perform emotion recognition based on the features of the EEG-fNIRS association matrix.

[0034] Optionally, the brain region division and electrode arrangement module specifically includes:

[0035] The brain region division submodule is used to divide the brain regions into the frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe according to the functions of different brain regions;

[0036] The electrode placement submodule is used to determine the type and arrangement order of electrodes placed in each of the defined brain regions.

[0037] Optionally, the electrode layout submodule specifically includes:

[0038] An electrode placement unit is used to place the EEG electrode in the frontal lobe region, the temporal lobe region, the central region, the parietal lobe region, and the occipital lobe region, and to place the fNIRS electrode in the frontal lobe region and the temporal lobe region; the fNIRS electrode and the EEG electrode do not overlap in the frontal lobe region and the temporal lobe region;

[0039] An electrode sequence arrangement unit is used to determine the electrode arrangement order of the EEG electrodes and the fNIRS electrodes for each of the brain regions.

[0040] Optionally, the electrode sequential arrangement unit specifically includes:

[0041] The interbrain electrode arrangement subunit is used to determine the electrode arrangement order of each of the divided brain regions, thereby obtaining the brain region electrode arrangement order.

[0042] The brain region electrode arrangement subunit is used to determine the EEG electrode arrangement order and fNIRS electrode arrangement order within each of the divided brain regions based on the electrode arrangement order between the brain regions.

[0043] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0044] This invention provides an EEG-fNIRS electrode arrangement method and system based on brain region segmentation. It simultaneously acquires EEG and fNIRS signals, leveraging their complementary advantages in temporal and spatial resolution to conduct electrode arrangement studies of EEG-fNIRS signals under multi-brain region segmentation. Taking into full account the differences in characteristics between EEG and fNIRS signals and the functional differences between different brain regions, a novel EEG-fNIRS signal electrode arrangement scheme is proposed. This scheme ensures that the constructed EEG-fNIRS correlation matrix fully reflects the interactions between different brain regions. This matrix is ​​then used as a feature input into a Convolutional Neural Network (CNN) for emotion classification, which is of great significance for a deeper understanding of the brain's neural activity mechanisms in emotion recognition tasks. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 The spatial positions of 62 EEG electrodes in the 10-20 international standard lead system provided by this invention;

[0047] Figure 2 This is a flowchart of an EEG-fNIRS electrode arrangement method based on brain region division provided in Embodiment 1 of the present invention;

[0048] Figure 3 This is a schematic diagram of the EEG-fNIRS electrode channel distribution provided in Embodiment 1 of the present invention;

[0049] Figure 4 A schematic diagram of the electrode arrangement for a 62-lead EEG and an 18-lead fNIRS provided in Embodiment 1 of the present invention;

[0050] Figure 5 The correlation matrix features constructed for the multi-brain region segmentation EEG-fNIRS electrode arrangement scheme provided in Embodiment 1 of the present invention;

[0051] Figure 6This is a structural diagram of the EEG-fNIRS synchronous acquisition device, which consists of a 62-lead EEG recorder and an 18-lead near-infrared brain functional imaging system, provided in Embodiment 1 of the present invention.

[0052] Figure 7 This is a schematic diagram showing the positional relationship between the fNIRS electrode and the EEG electrode in the EEG-fNIRS synchronous acquisition device provided in Embodiment 1 of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] The existing scheme for arranging nearest-neighbor EEG electrodes in the left and right hemispheres is implemented as follows:

[0055] (1) The EEG signal is preprocessed and the original EEG signal is decomposed into different frequency bands such as δ (0.5-4Hz), θ (4-8Hz), α (8-12Hz), β (12-30Hz) and γ (30-45Hz) using short-time Fourier transform;

[0056] (2) Calculation of EEG electrode connectivity characteristics. For EEG signals of various frequency bands, functional connectivity metrics are used to calculate the connectivity characteristics between pairs of EEG signals, such as Pearson correlation coefficient (PCC) and phase locking value (PLV).

[0057] The Pearson correlation coefficient (PCC) describes the linear relationship between two EEG signals x and y, and is calculated using the following formula:

[0058]

[0059] Where cov(·) represents the covariance, σ x and σ y These are the standard deviations of the two signals, respectively.

[0060] The phase-locked value (PLV) represents the phase synchronization of two EEG time series, and is expressed as the absolute average of the phase difference over a time window.

[0061]

[0062] Where N represents the number of windows, Δφ nThis represents the phase difference of the nth window.

[0063] (3) Division of left and right brain regions and electrode arrangement. Figure 1 The spatial distribution of 62 EEG electrodes in the international 10-20 system is shown. First, the EEG electrode signals are divided according to the left and right brain regions. Then, starting with the first electrode FP1 in the left frontal lobe region, the electrodes closest to the current electrode in the left brain region are selected as the next electrodes, until all electrodes in the left brain region are arranged. Finally, the same method is used to arrange the electrodes in the right brain region. The process ends at the central line.

[0064] (4) The EEG electrodes are rearranged according to the new electrode arrangement, and the connection features calculated from each pair of electrodes are arranged in the corresponding spatial positions to obtain a new EEG signal connection matrix, which is used by the CNN recognition network for emotion classification.

[0065] However, this existing electrode arrangement scheme only targets EEG signals and does not consider the impact of multi-brain region segmentation on electrode arrangement. Currently, no combined electrode arrangement scheme targeting both EEG and fNIRS signals has been discovered.

[0066] The interaction between different regions of the cerebral cortex plays a crucial role in the development of human behavior, higher psychological activities, and cognitive functions. One key research focus of EEG-fNIRS emotion recognition based on brain region segmentation, specifically targeting EEG and fNIRS signals acquired simultaneously during emotional video evoked emotions, is how to place and arrange EEG and fNIRS electrodes on the human head to better detect the brain regions' emotional response capabilities. Typically, the number of EEG electrodes is greater than that of fNIRS electrodes, and their placement is non-overlapping and complementary, each representing a different spatial location. Different electrode arrangements create different correlation matrices, inevitably impacting the performance of the emotion recognition system.

[0067] For EEG-fNIRS signals, most existing studies employ horizontal scanning, vertical scanning, or the shortest path electrode arrangement. Essentially, emotion is the result of the coordinated interaction of multiple brain regions in the cerebral cortex. However, most existing electrode arrangement schemes do not consider the impact of multi-region segmentation on electrode arrangement, and there is almost no coverage of electrode arrangement for simultaneously acquired EEG-fNIRS signals.

[0068] This invention focuses on EEG and fNIRS signals acquired synchronously during emotion-evoked video events. Leveraging the complementary advantages of their temporal and spatial resolutions, it conducts research on electrode arrangement for EEG-fNIRS signals under multi-brain region segmentation. Taking into full account the differences in characteristics between EEG and fNIRS signals and the functional differences between different brain regions, a novel EEG-fNIRS signal electrode arrangement scheme is proposed. This scheme ensures that the constructed EEG-fNIRS correlation matrix fully reflects the interactions between different brain regions. This matrix is ​​then used as a feature input into a Convolutional Neural Network (CNN) for emotion classification, which is of great significance for a deeper understanding of the brain's neural activity mechanisms in emotion recognition tasks.

[0069] The purpose of this invention is to provide an EEG-fNIRS electrode arrangement method and system based on brain region division. Taking into account the different categories of EEG signals and fNIRS signals, the different functions of different brain regions, and the similar functions of electrode nodes within the same brain region, a new EEG-fNIRS electrode arrangement method is designed so that the constructed spatiotemporally complementary connection matrix can better reflect the spatial information of EEG-fNIRS signals, thereby enabling more accurate emotion classification.

[0070] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0071] Example 1

[0072] One of the key focuses of EEG-fNIRS emotion recognition based on brain region segmentation is how to place and arrange EEG and fNIRS electrodes on the human head to better detect the brain regions' emotional response capabilities. This involves simultaneously acquiring emotional EEG-fNIRS signals, such as... Figure 2 As shown, this embodiment provides a method for EEG-fNIRS electrode arrangement based on brain region division, the method comprising:

[0073] S1: Using a 62-lead EEG recorder and an 18-lead near-infrared brain functional imaging system, the measured EEG signals and measured fNIRS signals of the brain during emotional cognitive activities were measured simultaneously (i.e., acquired synchronously).

[0074] Figure 6 The device shown is an EEG-fNIRS synchronous acquisition device consisting of a 62-lead EEG recorder and an 18-lead near-infrared brain functional imaging system. Figure 7 The positional relationship between the fNIRS electrode and the EEG electrode in the EEG-fNIRS synchronous acquisition device is shown.

[0075] Using emotional video clips as stimulation materials, a 62-lead EEG recorder and an 18-lead near-infrared brain functional imaging system were used to record the corresponding emotional EEG signals and fNIRS signals in the brain during emotional cognitive activities under different emotional evoked states.

[0076] EEG signals are voltage fluctuations generated during brain activity recorded on the scalp surface, directly reflecting the electrophysiological activity of neurons in the cerebral cortex. fNIRS electrodes are paired, one an emitter and the other a receiver, measuring changes in hemoglobin concentration in local brain tissue blood induced by neuronal activity. The electrode distribution for synchronously acquired EEG-fNIRS signals used in this invention is as follows: Figure 3 As shown, it includes a 62-lead EEG electrode channel and an 18-lead fNIRS electrode channel. Figure 3 The shaded circular area indicates the location of the fNIRS electrode, and the blank circular area indicates the location of the EEG electrode. Their locations do not overlap but are complementary. Specifically, in the left region of the frontal lobe, the fNIRS transmitting electrode S1 and the fNIRS receiving electrode D3 are positioned above and below the EEG electrode AF3. The fNIRS receiving electrode D1 is positioned to the right of the fNIRS transmitting electrode S1. The fNIRS transmitting electrode S3 and the fNIRS transmitting electrode S5 are positioned to the left and right of the fNIRS receiving electrode D3, respectively.

[0077] fNIRS transmitting electrode S1 is connected to fNIRS receiving electrode D1 and fNIRS receiving electrode D3 respectively; fNIRS transmitting electrode S5 is connected to fNIRS receiving electrode D1 and fNIRS receiving electrode D3 respectively; fNIRS transmitting electrode S3 is connected to fNIRS receiving electrode D3.

[0078] In the right region of the frontal lobe, fNIRS transmitting electrode S2 and fNIRS receiving electrode D4 are arranged above and below EEG electrode AF4, fNIRS receiving electrode D2 is arranged to the left of fNIRS transmitting electrode S2, and fNIRS transmitting electrode S4 and fNIRS transmitting electrode S6 are arranged to the right and left of fNIRS receiving electrode D4, respectively.

[0079] fNIRS transmitting electrode S2 is connected to fNIRS receiving electrode D2 and fNIRS receiving electrode D4 respectively; fNIRS transmitting electrode S6 is connected to fNIRS receiving electrode D2 and fNIRS receiving electrode D4 respectively; fNIRS transmitting electrode S4 is connected to fNIRS receiving electrode D4.

[0080] Within the left temporal lobe region, select any one of the aforementioned EEG electrodes (e.g. Figure 3 and Figure 4The selected EEG electrode (C5) is designated as the first target EEG electrode. Four fNIRS electrodes are arranged around the first target EEG electrode, designated as fNIRS emitting electrode S7, fNIRS emitting electrode S9, fNIRS receiving electrode D5, and fNIRS receiving electrode D7.

[0081] fNIRS transmitting electrode S7 is connected to fNIRS receiving electrode D5 and fNIRS receiving electrode D7 respectively; fNIRS transmitting electrode S9 is connected to fNIRS receiving electrode D5 and fNIRS receiving electrode D7 respectively.

[0082] Within the right temporal lobe region, select any one of the aforementioned EEG electrodes (e.g. Figure 3 and Figure 4 The selected EEG electrode (C6) is designated as the second target EEG electrode. Four fNIRS electrodes are arranged around the second target EEG electrode, designated as fNIRS emitting electrode S8, fNIRS emitting electrode S10, fNIRS receiving electrode D6, and fNIRS receiving electrode D8.

[0083] fNIRS transmitting electrode S8 is connected to fNIRS receiving electrode D6 and fNIRS receiving electrode D8, respectively; fNIRS transmitting electrode S10 is connected to fNIRS receiving electrode D6 and fNIRS receiving electrode D8, respectively.

[0084] Signal acquisition is followed by signal preprocessing:

[0085] The EEG signal and fNIRS signal obtained in step (1) are preprocessed respectively:

[0086] ① For the measured EEG signals, independent component analysis was first used to remove artifacts such as those from electrooculography (EOG), and an average reference was taken (the average value of data from all electrode channels was taken at the same time and used as the reference value for the 62 electrode channels at that time). The signals were then downsampled to 128 Hz and filtered from 0.5 to 45 Hz, with baseline correction performed using the signal 3 seconds before stimulation. Then, short-time Fourier transform was used to decompose the EEG signals into frequency bands, obtaining five different frequency bands: δ (0.5-4 Hz), θ (4-8 Hz), α (8-12 Hz), β (12-30 Hz), and γ (30-45 Hz).

[0087] ② For the measured fNIRS signal, the original acquired signal was first subjected to baseline correction, artifact removal, and a 0.01-0.2Hz bandpass filter. The changes in oxyhemoglobin (HbO2) and deoxyhemoglobin (HbR) concentrations were then calculated using a modified Beer-Lambert law. Next, the fNIRS signal was upsampled at 128Hz to match the EEG signal in the time domain.

[0088] S2: Divide the brain into multiple regions and determine the type and arrangement order of electrodes in each region to obtain the EEG-fNIRS electrode arrangement for each region. The electrode types in each region include EEG electrodes and fNIRS electrodes. All EEG electrodes correspond one-to-one with the physical EEG electrodes in the 62-lead EEG recorder. All fNIRS electrodes correspond one-to-one with the physical fNIRS electrodes in the 18-lead near-infrared brain functional imaging system.

[0089] The human brain can be divided into five regions according to their functions: the frontal lobe, temporal lobe, central region, parietal lobe, and occipital lobe. Figure 4 As shown in the figure, EEG and fNIRS signals are distributed in the frontal and temporal lobes. In addition, EEG signals are also distributed in the central region, parietal lobe, and occipital lobe.

[0090] Based on the categories of EEG and fNIRS signals and the characteristics of brain regions, this invention arranges the EEG and fNIRS brain regions independently. First, the EEG brain regions are arranged in a "top-down" manner, that is, in the order of "frontal lobe-temporal lobe-central region-parietal lobe-occipital lobe". Then, the fNIRS brain regions are arranged in the order of "frontal lobe-temporal lobe".

[0091] Specifically, step S2 includes:

[0092] S21: The brain regions are divided into the frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe according to their functions.

[0093] S22: Determine the type of electrodes and the order of electrode arrangement in each of the described brain regions.

[0094] Specifically, step S22 includes:

[0095] S22-1: The EEG electrode is disposed in the frontal lobe region, the temporal lobe region, the central region, the parietal lobe region, and the occipital lobe region; the fNIRS electrode is disposed in the frontal lobe region and the temporal lobe region; the fNIRS electrode and the EEG electrode do not overlap in the frontal lobe region and the temporal lobe region.

[0096] S22-2: Determine the electrode arrangement order of the EEG electrodes and the fNIRS electrodes for each of the brain regions.

[0097] Specifically, step S22-2 includes:

[0098] S22-2(1): Determine the electrode arrangement order of each of the brain regions to obtain the brain region electrode arrangement order.

[0099] S22-2(2): Based on the electrode arrangement order of the brain regions, determine the EEG electrode arrangement order and fNIRS electrode arrangement order within each of the divided brain regions.

[0100] Specifically, step S22-2(2) includes:

[0101] (2-1) The order of the brain regions of the EEG electrodes is obtained by following the arrangement order of the frontal lobe, the temporal lobe, the central region, the parietal lobe and the occipital lobe.

[0102] (2-2) The arrangement order of the fNIRS electrodes in the brain regions is determined sequentially according to the order of the frontal lobe region first and then the temporal lobe region, specifically including:

[0103] (2-2-1) For each of the brain regions, a target electrode is randomly selected as the selection electrode; when the EEG electrode arrangement order is determined, the target electrode is the EEG electrode at the boundary of the brain region to which it belongs; when the fNIRS electrode arrangement order is determined, the target electrode is the fNIRS electrode at the boundary of the brain region to which it belongs.

[0104] (2-2-2) Based on the principle of left first and right second, select the electrode that is closest to the selected electrode as the next target electrode.

[0105] (2-2-3) Using the next target electrode as the selected electrode, return to step (2-2-2) "select the electrode closest to the selected EEG electrode as the next target electrode based on the principle of left first and right first", until the target electrodes of each brain region are arranged.

[0106] from Figure 4 As can be seen, the electrodes within each EEG and fNIRS brain region are symmetrically distributed in the left and right hemispheres. Therefore, this invention utilizes the functional lateralization of the left and right brain regions to propose a "left-first, right-second" electrode arrangement method within each EEG and fNIRS brain region. The final electrode distribution of EEG and fNIRS signals under multi-brain region segmentation is shown in Table 1.

[0107] Table 1. Electrode arrangement order under multiple brain region segmentation

[0108]

[0109] S3: Based on the measured EEG signals and measured fNIRS signals of each frequency band and the EEG-fNIRS electrode arrangement of each brain region, determine the EEG-fNIRS correlation matrix features of the EEG signals and fNIRS signals of each brain region.

[0110] Signal preprocessing yields EEG signals in five frequency bands, which in turn constructs five EEG-fNIRS correlation matrix features.

[0111] Based on the designed EEG-fNIRS electrode arrangement and the measured EEG and fNIRS signals, the correlation between each pair of electrode channels is calculated using functional connectivity metrics such as PLV and PCC, or effectiveness metrics such as Granger causality (GC) and transfer entropy. The EEG-fNIRS correlation matrix features are then constructed, and its structure is as follows: Figure 5 As shown, the EEG-fNIRS correlation matrix constructed based on the multi-brain region electrode arrangement scheme proposed in this invention can be divided into two categories: one is the correlation within each brain region, such as... Figure 5 The shaded areas in the middle and lower reaches of the brain are shown; another type is the pairwise correlation between EEG and fNIRS brain regions, such as... Figure 5 The shaded area is shown in the image.

[0112] S4: Perform sentiment recognition based on the features of the EEG-fNIRS association matrix.

[0113] This invention primarily utilizes synchronously acquired emotional EEG and fNIRS signals to investigate the potential synergistic interactions between brain regions during emotion processing. It explores the inter-class differences between EEG and fNIRS signals by arranging them independently; it also explores the inter-brain information interaction of EEG-fNIRS signals by arranging them sequentially across multiple brain regions; and it designs an "left-brain-first, right-brain-later" electrode arrangement within brain regions, leveraging the functional lateralization of the human brain, to explore the impact of brain region asymmetry on emotion. The main technical problem this invention aims to solve is how to utilize the categories and brain region characteristics of EEG and fNIRS signals to study EEG-fNIRS electrode arrangement methods under multi-brain region segmentation, and construct a corresponding EEG-fNIRS correlation matrix to effectively improve emotion recognition performance.

[0114] Example 2

[0115] This embodiment provides an EEG-fNIRS electrode arrangement system based on brain region division, the system comprising:

[0116] The signal acquisition module M1 is used to simultaneously acquire measured EEG signals and measured fNIRS signals of the brain during emotional cognitive activities using a 62-lead electroencephalogram (EEG) recorder and an 18-lead near-infrared brain functional imaging system.

[0117] The brain region division and electrode arrangement module M2 is used to divide the brain into multiple regions and determine the electrode types and arrangement order in each divided brain region to obtain the EEG-fNIRS electrode arrangement for each divided brain region. The electrode types in each divided brain region include EEG electrodes and fNIRS electrodes. All EEG electrodes correspond one-to-one with the physical EEG electrodes in the 62-lead EEG recorder. All fNIRS electrodes correspond one-to-one with the physical fNIRS electrodes in the 18-lead near-infrared brain functional imaging system.

[0118] The brain region division and electrode arrangement module specifically includes:

[0119] The brain region division submodule is used to divide the brain regions into the frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe according to the functions of different brain regions.

[0120] The electrode placement submodule is used to determine the type and arrangement order of electrodes placed in each of the defined brain regions.

[0121] The electrode placement submodule specifically includes:

[0122] An electrode placement unit is used to place the EEG electrode in the frontal lobe region, the temporal lobe region, the central region, the parietal lobe region, and the occipital lobe region, and to place the fNIRS electrode in the frontal lobe region and the temporal lobe region; the fNIRS electrode and the EEG electrode do not overlap in the frontal lobe region and the temporal lobe region.

[0123] An electrode sequence arrangement unit is used to determine the electrode arrangement order of the EEG electrodes and the fNIRS electrodes for each of the brain regions.

[0124] The electrode sequential arrangement unit specifically includes:

[0125] The interbrain electrode arrangement subunit is used to determine the electrode arrangement order of each of the divided brain regions, thereby obtaining the brain region electrode arrangement order.

[0126] The brain region electrode arrangement subunit is used to determine the EEG electrode arrangement order and fNIRS electrode arrangement order within each of the divided brain regions based on the electrode arrangement order between the brain regions.

[0127] The correlation matrix feature extraction module M3 is used to determine the EEG-fNIRS correlation matrix features of the EEG signals and fNIRS signals of each brain region based on the measured EEG signals, the measured fNIRS signals, and the EEG-fNIRS electrode arrangement of each brain region.

[0128] The emotion recognition module M4 is used to perform emotion recognition based on the features of the EEG-fNIRS association matrix.

[0129] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0130] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for EEG-fNIRS electrode arrangement based on brain region division, characterized in that, The method includes: The measured EEG signals and measured fNIRS signals of the brain during emotional cognitive activities were simultaneously acquired using a 62-lead EEG recorder and an 18-lead near-infrared brain functional imaging system. The brain regions are divided into frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe according to their functions. The types and order of electrodes deployed in each region are determined to obtain the EEG-fNIRS electrode layout for each region. The electrode types deployed in each region include EEG electrodes and fNIRS electrodes. All EEG electrodes correspond one-to-one with the physical EEG electrodes deployed in the 62-lead EEG recorder. All fNIRS electrodes correspond one-to-one with the physical fNIRS electrodes deployed in the 18-lead near-infrared brain functional imaging system. Based on the measured EEG signals and the measured fNIRS signals, as well as the EEG-fNIRS electrode arrangement of each brain region, the correlation between pairs of electrode channels is measured to determine the EEG-fNIRS correlation matrix features of the EEG signals and fNIRS signals of each brain region; the EEG-fNIRS correlation matrix features include the correlation within each brain region and the correlation between pairs of EEG and fNIRS brain regions. Sentiment recognition is performed based on the features of the EEG-fNIRS association matrix. This includes determining the type and arrangement order of electrodes deployed in each brain region, specifically including: The EEG electrodes are disposed in the frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe, and the fNIRS electrodes are disposed in the frontal lobe and temporal lobe; the fNIRS electrodes and the EEG electrodes do not overlap in the frontal lobe and temporal lobe. The order of the EEG electrodes in the brain regions is determined by sequentially arranging the frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe regions; the order of the fNIRS electrodes in the brain regions is determined by sequentially arranging the frontal lobe regions first and then the temporal lobe regions. For each of the defined brain regions, a target electrode is randomly selected as the selection electrode; when the EEG electrode arrangement order is determined, the target electrode is the EEG electrode at the boundary of the defined brain region; when the fNIRS electrode arrangement order is determined, the target electrode is the fNIRS electrode at the boundary of the defined brain region. Based on the principle of left-to-right selection, the electrode closest to the selected electrode is chosen as the next target electrode; Using the next target electrode as the selected electrode, return to the step "select the electrode closest to the selected electrode as the next target electrode based on the principle of left to right" until all target electrodes for the current brain region division are arranged.

2. An EEG-fNIRS electrode arrangement system based on brain region division, characterized in that, The system includes: The signal acquisition module is used to simultaneously acquire measured EEG signals and measured fNIRS signals of the brain during emotional cognitive activities using a 62-lead EEG recorder and an 18-lead near-infrared brain functional imaging system. The brain region division and electrode arrangement module is used to divide the brain into multiple regions and determine the electrode types and arrangement order in each divided brain region, thus obtaining the EEG-fNIRS electrode arrangement for each divided brain region. The electrode types deployed in each divided brain region include EEG electrodes and fNIRS electrodes. All EEG electrodes correspond one-to-one with the physical EEG electrodes deployed in the 62-lead EEG recorder. All fNIRS electrodes correspond one-to-one with the physical fNIRS electrodes deployed in the 18-lead near-infrared brain functional imaging system. The brain region division and electrode arrangement module includes: The brain region division submodule is used to divide the brain regions into the frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe according to the functions of different brain regions; The electrode placement submodule is used to determine the type of electrodes and the order of electrode placement in each of the divided brain regions. The electrode placement submodule specifically includes: An electrode placement unit is used to place the EEG electrode in the frontal lobe region, the temporal lobe region, the central region, the parietal lobe region, and the occipital lobe region, and to place the fNIRS electrode in the frontal lobe region and the temporal lobe region; the fNIRS electrode and the EEG electrode do not overlap in the frontal lobe region and the temporal lobe region; An electrode sequence arrangement unit is used to determine the electrode arrangement order of the EEG electrodes and the fNIRS electrodes for each brain region segmentation. The electrode sequential arrangement unit specifically includes: The interbrain electrode arrangement subunit is used to determine the electrode arrangement order of each of the divided brain regions, thereby obtaining the brain region electrode arrangement order. The brain region electrode arrangement subunit is used to determine the EEG electrode arrangement order and fNIRS electrode arrangement order within each of the divided brain regions based on the electrode arrangement order between the brain regions. Specifically, determining the electrode arrangement order for each of the defined brain regions to obtain the brain region electrode arrangement order includes: The EEG electrodes are disposed in the frontal lobe, temporal lobe, central lobe, parietal lobe, and occipital lobe, and the fNIRS electrodes are disposed in the frontal lobe and temporal lobe; the fNIRS electrodes and the EEG electrodes do not overlap in the frontal lobe and temporal lobe. Specifically, determining the EEG electrode arrangement order and fNIRS electrode arrangement order within each of the defined brain regions includes: determining the brain region electrode arrangement order of the EEG electrodes according to the arrangement order of the frontal lobe region, the temporal lobe region, the central region, the parietal lobe region, and the occipital lobe region; and determining the brain region electrode arrangement order of the fNIRS electrodes according to the arrangement order of the frontal lobe region first and then the temporal lobe region. For each of the defined brain regions, a target electrode is randomly selected as the selection electrode; when the EEG electrode arrangement order is determined, the target electrode is the EEG electrode at the boundary of the defined brain region; when the fNIRS electrode arrangement order is determined, the target electrode is the fNIRS electrode at the boundary of the defined brain region. Based on the principle of left-to-right selection, the electrode closest to the selected electrode is chosen as the next target electrode; Using the next target electrode as the selected electrode, return to the step "select the electrode closest to the selected electrode as the next target electrode based on the principle of left to right" until all target electrodes for the current brain region division are arranged. The correlation matrix feature extraction module is used to determine the EEG-fNIRS correlation matrix features of the EEG signals and fNIRS signals of each brain region based on the measured EEG signals and the measured fNIRS signals, as well as the EEG-fNIRS electrode arrangement of each brain region. The EEG-fNIRS correlation matrix features include the correlation within each brain region and the correlation between each pair of EEG and fNIRS brain regions. The emotion recognition module is used to perform emotion recognition based on the features of the EEG-fNIRS association matrix.

Images