High-density EEG-fNIRS attention monitoring system based on AFHRF

Abstract

The invention discloses a high-density EEG-fNIRS attention monitoring system based on an AFHRF (Attention Free High Frequency Radio Frequency). The system comprises an EEG-fNIRS data acquisition and preprocessing module which is used for acquiring original multi-channel EEG-fNIRS data and preprocessing the original data; the information intensive processing module is used for carrying out feature dimension reduction on the EEG-fNIRS data on the basis of AFHRF and reducing the complexity requirement and the sample size requirement of a subsequent feature network; the spatio-temporal feature extraction module is used for extracting spatio-temporal features related to tasks in the EEG-fNIRS data subjected to information densification processing and identifying changes of attention; and the classification result acquisition module outputs an identification result of a corresponding moment and judges attention change. According to the method, before the data are imported into the feature network, the AFHRF is innovatively adopted to extract the data correlation features and reduce the dimensions, so that the efficiency and the practicability of the system are remarkably improved while the monitoring accuracy is ensured.

Description

Technical Field

[0001] The present invention relates to the field of neuroscience, and particularly to a high-density EEG-fNIRS attention monitoring system based on an adaptive hemodynamic response function (AFHRF). Background Art

[0002] Existing EEG-fNIRS attention monitoring systems still face many challenges in practical applications. On the one hand, the original EEG-fNIRS data is characterized by high dimensionality and high noise, containing a large amount of redundant information irrelevant to attention monitoring. When inputting this data into a feature network for analysis, the high-dimensional data will lead to a significant increase in the complexity of the feature network and a sharp rise in the computational cost. It not only consumes a large amount of computing resources and time but also easily causes overfitting problems, resulting in poor performance of the model on new data. On the other hand, in order to train an accurate and reliable feature network, a large amount of sample data is often required to learn the feature patterns in the data. However, in actual operation, it is difficult to obtain a large amount of high-quality EEG-fNIRS sample data, such as high experimental costs and difficulties in recruiting subjects, which limits the scalability and practicality of the system.

[0003] The hemodynamic response function describes the process from the start of a stimulus to the peak of the hemodynamic response and then back to the baseline. By analyzing the hemodynamic response function, the brain's activity patterns under different cognitive tasks can be understood, such as vision, audition, language, memory, etc.; the hemodynamic response function also has important significance in the study of brain diseases, which can help identify abnormal activities in the nervous system, such as obsessive-compulsive disorder, autism spectrum disorder, post-traumatic stress disorder, and brain injury, etc.; matching the events in the design matrix with the hemodynamic response function through convolution operations to predict the brain's activity pattern is of great significance for improving the accuracy of brain-computer interfaces.

[0004] However, the hemodynamic response function calculated based on the event matrix has limited temporal resolution and cannot capture rapidly changing neural activities. For example, Tan Sen used the event-related component of the EEG signal to replace the traditional event matrix in the invention patent CN118845016A to improve the accuracy of fatigue detection. However, the calculation of the hemodynamic response function is based on a linear assumption, but in fact, brain activities may be affected by various non-linear factors, such as the release of neurotransmitters and vascular responses. This analysis method still has a high error. Existing data processing methods are insufficient in exploring the internal correlation between EEG and fNIRS data and fail to fully utilize the complementary information of the two modalities of data. The traditional hemodynamic response function (HRF) is usually based on fixed model assumptions and cannot be adaptively adjusted according to different experimental scenarios and individual differences, resulting in certain limitations in extracting the correlation features of EEG and fNIRS data and affecting the accuracy and reliability of attention monitoring.

[0005] Therefore, it is of great practical significance and application value to develop a new type of attention monitoring system that can effectively reduce the data dimension, reduce the sample requirement, and fully explore the correlation features of EEG and fNIRS data. Summary of the Invention

[0006] The present invention provides a high-density EEG-fNIRS attention monitoring system based on AFHRF, which can effectively reduce the data dimension, reduce the sample requirement, and fully explore the correlation features of EEG and fNIRS data. While ensuring the monitoring accuracy, it significantly improves the efficiency and practicality of the system.

[0007] The high-density EEG-fNIRS attention monitoring system based on AFHRF includes:

[0008] An EEG-fNIRS data acquisition and preprocessing module, which is used to acquire the original multi-channel EEG-fNIRS data and preprocess the original data;

[0009] An information intensification processing module, which performs feature dimension reduction on the EEG-fNIRS data based on AFHRF to reduce the complexity requirement and sample size requirement of the subsequent feature network;

[0010] A spatio-temporal feature extraction module, which uses temporal convolution kernels and spatial convolution kernels to extract the task-related spatio-temporal features in the EEG-fNIRS data after information intensification processing to identify changes in attention;

[0011] A classification result acquisition module, which outputs the recognition result at the corresponding moment to determine the change in attention.

[0012] The information intensification processing module performs feature dimensionality reduction on the preprocessed EEG-fNIRS data based on AFHRF. By mining the correlation features between EEG and fNIRS data, it effectively reduces the dimensionality of the features to be processed by the subsequent feature network, thereby reducing the complexity requirements of the feature network and the requirements for the sample size.

[0013] Preferably, the information intensification processing module specifically includes: an AFHRF generation module and an EEG-fNIRS fusion feature extraction module. After the EEG-fNIRS data is generated based on the AFHRF function generated by the AFHRF generation module, the cross-modal correlation features of the two are extracted by the EEG-fNIRS fusion feature extraction module.

[0014] Preferably, the AFHRF generation module specifically includes:

[0015] A sample data acquisition and preprocessing module, in which the subject completes a nerve activation task, synchronously acquires and records the subject's EEG and fNIRS data and preprocesses the sample data;

[0016] A feature extraction network training module, using the synchronously acquired fNIRS data as labels to train the EEG feature extraction feature network;

[0017] An AFHRF calculation module, generating simulated fNIRS data through the feature extraction network and calculating the adaptive hemodynamic response function by deconvolution.

[0018] Preferably, the feature extraction network training module uses a spatio-temporal feature convolutional network for EEG feature extraction, reduces the dimension to be the same as the fNIRS data through a fully connected layer, and calculates the loss function through the mean square error.

[0019] Preferably, the sample data is divided into a training set and a test set according to a certain ratio. The feature extraction network training module uses the fNIRS data synchronously acquired from the training set as labels to train the feature extraction network for feature extraction of the EEG data corresponding to the channels in the training set.

[0020] Preferably, the AFHRF calculation module imports the EEG data in the test set into the feature extraction network to generate simulated fNIRS data, uses the simulated fNIRS data as the data after convolution, and the real EEG data corresponding to the channels as the event matrix, and calculates the adaptive hemodynamic response function by deconvolution.

[0021] Preferably, the AFHRF generation module further includes an AFHRF accuracy test module. After importing the EEG data in the test set into the feature extraction network to generate simulated fNIRS data, it further includes comparing the simulated fNIRS data with the real fNIRS data to test the accuracy of the simulated fNIRS data.

[0022] Preferably, the time convolution kernel of the spatio-temporal feature convolution network has the following mathematical expression:

[0023] X' = Conv1d ( X, W t ) + b t

[0024] Where: X is the input EEG data, with the shape of (N, C, T), N is the number of samples, C is the number of channels (i.e., the number of electrodes), and T is the time step; Wt is the time convolution kernel, with the shape of (Cout, 1, 1), where Cout is the number of output channels; bt is the bias term, with the shape of (Cout, 1, 1); X′ is the output after convolution, with the shape of (N, Cout, T).

[0025] Preferably, the space convolution kernel of the spatio-temporal feature convolution network has the following mathematical expression:

[0026] Y = Conv2d ( X, W s ) + b s

[0027] Where: Ws is the space convolution kernel, with the shape of (Tout, Cout, 1, 1), Tout is the time step, Cout is the number of output channels; bs is the bias term, with the shape of (Tout, Cout); Y is the output after convolution, with the shape of (N, Tout, Cout).

[0028] Compared with the prior art, the high-density EEG-fNIRS attention monitoring system based on AFHRF provided by the present invention can at least achieve the following beneficial effects:

[0029] By reducing the dimensionality of data features through AFHRF, redundant information is effectively removed, the amount of calculation and processing time are reduced, the real-time performance of the system is improved, and at the same time, the dependence on the sample size is reduced, and the experimental cost and the difficulty of data collection are reduced; AFHRF can adaptively mine the correlation features of EEG and fNIRS data, combine the advantages of both, and avoid the limitations of a single modality. The spatio-temporal convolution kernel accurately extracts spatio-temporal features related to the task, improves the accuracy of attention monitoring; reduces the complexity of the feature network, reduces the risk of overfitting, enhances the generalization and stability of the system in different scenarios and populations, and has a wide application prospect. Brief Description of the Drawings

[0030] Figure 1 It is a schematic structural diagram of a high-density EEG-fNIRS attention monitoring system based on AFHRF provided by an embodiment of the present invention.

[0031] Figure 2 It is an overall flowchart for the AFHRF generation module provided by an embodiment of the present invention to generate an adaptive hemodynamic response function.

[0032] Figure 3 It is a detailed flowchart of the EEG-fNIRS data synchronous acquisition task of the sample data acquisition and preprocessing module provided by an embodiment of the present invention.

[0033] Figure 4 It is a detailed flowchart of the EEG-fNIRS data preprocessing and segmentation of the sample data acquisition and preprocessing module provided by an embodiment of the present invention.

[0034] Figure 5 It is a feature network framework of the feature extraction network training module provided by an embodiment of the present invention. Detailed Embodiments

[0035] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0036] Please refer to Figure 1 , Figure 1 as, a schematic structural diagram of a high-density EEG-fNIRS attention monitoring system based on AFHRF.

[0037] The present invention provides a high-density EEG-fNIRS attention monitoring system based on AFHRF, which can effectively reduce the data dimension, reduce the sample requirement, fully explore the correlation features of EEG and fNIRS data, and significantly improve the efficiency and practicality of the system while ensuring the monitoring accuracy. The high-density EEG-fNIRS attention monitoring system based on AFHRF includes: an EEG-fNIRS data acquisition and preprocessing module, which is used to acquire the original multi-channel EEG-fNIRS data and preprocess the original data; an information densification processing module, which performs feature dimension reduction on the EEG-fNIRS data based on AFHRF to reduce the complexity requirement and sample size requirement of the subsequent feature network; a spatio-temporal feature extraction module, which uses a temporal convolution kernel and a spatial convolution kernel to extract the task-related spatio-temporal features in the EEG-fNIRS data after information densification processing to identify the change of attention; and a classification result acquisition module, which outputs the recognition result at the corresponding moment to determine the change of attention.

[0038] It can be understood that the hemodynamic response function can describe the time course of hemodynamic changes caused by neural activities. In the fusion analysis of EEG and fNIRS data, it can establish a connection between the neural electrical activities represented by EEG and the hemodynamic changes represented by fNIRS. For example, when neural activities occur in a certain area of the brain, the hemodynamic changes in that area will be caused after a certain time, and the hemodynamic response function can capture this relationship from neural activities to blood flow changes, so as to extract the correlation features between EEG and fNIRS data. The high-density EEG-fNIRS attention monitoring system based on AFHRF provided by the embodiments of the present invention generates AFHRF by building an information densification processing module, extracts the correlation features that to a certain extent represent the key information in the original EEG and fNIRS data through the generated AFHRF, and uses these correlation features to replace the original high-dimensional data, thereby realizing the reduction of the feature dimension. When the data dimension input into the feature network of the spatio-temporal feature extraction module is reduced, the amount of information that the feature network needs to process becomes less, and the complexity of the network will naturally decrease. For example, a complex network structure that originally needed to process a large amount of EEG and fNIRS original data can be simplified into a smaller-scale network after using the correlation features, reducing the number of parameters and the amount of calculation in the network.

[0039] The embodiments of the present invention specifically describe the AFHRF generation module in the information densification processing module of the present invention. The content of other modules can be understood by those skilled in the relevant art through the existing description, and will not be elaborated here. The AFHRF generation module specifically includes:

[0040] Sample data acquisition and preprocessing module, where the subject completes a neural activation task, synchronously acquires and records the subject's EEG and fNIRS data, and preprocesses the sample data;

[0041] Feature extraction network training module, using the synchronously acquired fNIRS data as labels to train the EEG feature extraction network;

[0042] AFHRF calculation module, generating simulated fNIRS data through the feature extraction network and deconvolving to calculate the adaptive hemodynamic response function.

[0043] Specifically, the AFHRF generation module generates the adaptive hemodynamic response function for the fused EEG and fNIRS data through the following method:

[0044] Please refer to Figure 2 , such as Figure 2 shown in the overall flowchart of the AFHRF generation module provided by the embodiment of the present invention for generating the adaptive hemodynamic response function. It mainly includes the following steps: synchronously acquiring and recording EEG and fNIRS data; respectively preprocessing the EEG and fNIRS data at the same time to obtain the preprocessed EEG and fNIRS data; training the feature network to extract features from the EEG data, with the fNIRS data as labels; generating simulated fNIRS data through feature network inference and deconvolving to calculate the adaptive hemodynamic response function.

[0045] As a specific embodiment of the present invention, the specific steps for generating the adaptive hemodynamic response function specifically include:

[0046] Step 1: Synchronously acquire and record the subject's EEG and fNIRS data, and respectively preprocess the EEG and fNIRS data at the same time.

[0047] As a specific embodiment of the present invention, first, the subject completes a neural activation task, and then the multi-channel EEG-fNIRS device synchronously acquires and records the subject's EEG and fNIRS data.

[0048] It can be understood that the multi-channel EEG-fNIRS device needs to be equipped with EEG electrodes and fNIRS probes at the same time. Corresponding to the same channel site, the EEG electrode patch and the fNIRS probe overlap, and can synchronously acquire the EEG and fNIRS signals at the same site in time and space.

[0049] Specifically, the nerve activation task aims to stimulate the neuronal activity of the brain through specific cognitive or motor activities, thereby generating detectable electrophysiological signals and near-infrared spectroscopy signals. The nerve activation task may include: cognitive tasks, such as mental arithmetic tasks, language comprehension tasks, memory recall tasks, etc. These tasks require the subject to perform high-intensity cognitive processing to activate the neuronal activity in related regions of the brain.

[0050] As an implementation manner of the present invention, the nerve activation task in this embodiment is specifically a cognitive task. Refer to Figure 3 , Figure 3 The detailed flowchart of the EEG-fNIRS data synchronous acquisition task of the sample data acquisition and preprocessing module provided by the embodiment of the present invention. Further, based on the understanding of the hemodynamic principle and the feature network training process, this embodiment guides the subject to alternately perform mental arithmetic tasks and resting tasks. The mental arithmetic lasts for 10 s, the rest lasts for 10 s, and one alternation is a complete event, so that the blood flow can complete a complete adjustment. To ensure that the sample size required for training is collected, and at the same time avoid excessive execution rounds from causing subject fatigue and reducing the accuracy of the feature network, as one of the implementation manners of the present invention, 20 rounds of the complete event are executed.

[0051] As one of the implementation manners of the present invention, the EEG-fNIRS synchronous acquisition device worn by the subject needs to be synchronized both spatially and temporally. It can be understood that spatial synchronization specifically refers to the spatial synchronization of the multi-channel EEG-fNIRS device, that is, for the same site, the EEG electrode patch and the fNIRS probe are nearly overlapped. Temporal synchronization specifically means that the acquisition processes of the two signals are synchronized, and it is not possible to collect EEG once and then collect fNIRS once. The multi-channel EEG-fNIRS device described in the embodiment of the present invention is provided with EEG electrodes and fNIRS probes at the same time. For the same channel site, the EEG electrode patch and the fNIRS probe overlap, and can synchronously collect the EEG and fNIRS signals at the same site in terms of time and space.

[0052] As one of the implementation manners of the present invention, the EEG sampling rate of the EEG-fNIRS synchronous acquisition device worn by the subject is 1000 Hz, and the fNIRS sampling rate is 50 Hz.

[0053] Step 1 further includes: respectively preprocessing the EEG and fNIRS data at the same time to obtain the preprocessed EEG and fNIRS data.

[0054] Please refer to Figure 4 , as Figure 4 shown is the detailed flowchart of the EEG-fNIRS data preprocessing and segmentation of the sample data acquisition and preprocessing module provided by the embodiment of the present invention.

[0055] As one of the embodiments of the present invention, both EEG and fNIRS need to be preprocessed to reduce physiological and environmental interferences, and the data is segmented into independent samples according to a preset time step.

[0056] Based on the understanding of the processing of EEG and fNIRS data, as one of the embodiments of the present invention, the baseline of EEG data and fNIRS data is removed using empirical mode decomposition. This is because the electrodes and probes cannot be fully attached to the skin surface, and there is inevitably an irregular baseline. Empirical mode decomposition can decompose the data into different components, and the remaining residual is the baseline. The EEG data is processed using a Butterworth band-pass filter of 0.5 Hz - 40 Hz. In most environments, the 50 Hz power frequency interference is difficult to avoid, and in the neural activities activated by cognitive tasks, the highest frequency cutoff is at 30 Hz, the highest frequency of β2. Therefore, the 40 Hz low-pass filtering retains all the information that needs to be concerned while filtering out high-frequency interferences. The fNIRS data is first converted from light intensity data to blood oxygen data and then processed. Interferences outside (0.01, 0.1) Hz (including heartbeat, respiration, etc.) are filtered out by band-pass, and outliers are removed using median filtering. As one of the embodiments of the present invention, the data segmentation is performed according to the event length, and the data is segmented into samples of 20 s each, with a step size of 2 s.

[0057] Step 2: Using the synchronously acquired fNIRS data as labels, train the EEG feature extraction feature network.

[0058] As one of the embodiments of the present invention, specifically, a spatio-temporal feature convolutional network is used to extract the features of EEG signals, and the dimension is reduced through a fully connected layer to be the same as that of fNIRS data. fNIRS is used as a label, and the loss function is calculated through mean square error.

[0059] Furthermore, a certain proportion of the EEG data of all channels and the corresponding site fNIRS samples obtained through Step 1 are selected as the training set to train the feature network, and the remaining samples are used as the test set to be imported into the feature network to infer and generate simulated fNIRS data, and the adaptive hemodynamic response function is calculated through deconvolution. As one of the embodiments of the present invention, 90% of the samples are used as the training set, and the remaining 10% of the samples are used as the test set.

[0060] Please refer to Figure 5 as Figure 5The following shows the feature network framework of the feature extraction network training module provided by the embodiment of the present invention. After the multi-channel EEG data in the training set samples is imported into the feature extraction network, the shape is first changed to adapt to the dimension of the temporal convolution kernel. The temporal convolution kernel is a one-dimensional convolution kernel that only convolves in the temporal dimension. The mathematical expression of the temporal convolution kernel is as follows:

[0061] X' = Conv1d(X, W t ) + b t

[0062] Where: X is the input EEG data, with the shape of (N, C, T), N is the number of samples, C is the number of channels (i.e., the number of electrodes), and T is the time step; Wt is the temporal convolution kernel, with the shape of (Cout, 1, 1), where Cout is the number of output channels; bt is the bias term, with the shape of (Cout, 1, 1). X′ is the output after convolution, with the shape of (N, Cout, T).

[0063] The spatial convolution kernel convolves in the channel dimension and focuses on the features between channels. The mathematical expression of the spatial convolution kernel is as follows:

[0064] Y = Conv2d(X, W s ) + b s

[0065] Where: Ws is the spatial convolution kernel, with the shape of (Tout, Cout, 1, 1), Tout is the time step, Cout is the number of output channels; bs is the bias term, with the shape of (Tout, Cout). Y is the output after convolution, with the shape of (N, Tout, Cout).

[0066] The last layer is a fully connected layer, and the output dimension is the same as that of the labeled fNIRS data. The mean squared error is selected as the loss function, and the calculation is transformed into a regression problem.

[0067] Based on the test of this feature extraction network, as one of the embodiments of the present invention, the shape of the temporal convolution kernel is (1, 64), the number is 8, and the shape of the spatial convolution kernel is (C, 1), and the number is 4.

[0068] Step 3: Generate simulated fNIRS data through the feature extraction network, and calculate the adaptive hemodynamic response function by deconvolution.

[0069] As one of the embodiments of the present invention, when generating an adaptive hemodynamic response function, the fNIRS data simulated by the feature extraction network is used as the data after convolution, and the EEG data of the corresponding channels is used as the event matrix, and a one-dimensional hemodynamic response function is derived and calculated. After the test set EEG data is imported into the feature network, the generated simulation data needs to be compared with the real fNIRS data for accuracy testing. As one of the embodiments of the present invention, the average accuracy within 20 s is tested. When the accuracy exceeds 70%, the training is stopped; otherwise, the data volume is amplified and steps 1-2 are repeated.

[0070] To improve the generalization of the attention recognition model, the embodiment of the present invention divides the attention recognition network into two modules, an attention spatio-temporal feature extraction module and a classification result acquisition module. The attention spatio-temporal feature extraction module uses the same spatio-temporal feature extraction network as the information densification processing module, and the output dimension is (1, 15). The classification result acquisition module consists of a Focallose loss function and a three-layer multi-layer perceptron. The Focallose loss function is directly connected to the attention spatio-temporal feature extraction module. During training, each training first completes the training of the attention spatio-temporal feature extraction module, and then completes the training of the information densification processing module. The two trainings are isolated and alternately trained. The classification result acquisition module outputs the attention result, which includes both the cognitive event category and the probability of the corresponding activation event, and is used for subsequent precise intervention.

[0071] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0072] According to the disclosure and teachings of the above specification, those skilled in the art to which the present invention pertains can also make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the protection scope of the claims of the present invention. According to the disclosure and teachings of the above specification, those skilled in the art to which the present invention pertains can also make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the protection scope of the claims of the present invention.

Claims

1. A high-density EEG-fNIRS attention monitoring system based on AFHRF, characterized in that: include: EEG-fNIRS data acquisition and preprocessing module, used to acquire raw multi-channel EEG-fNIRS data and preprocess the raw data; An information density processing module performs feature dimension reduction on the EEG-fNIRS data based on AFHRF to reduce the complexity and sample size requirements of subsequent feature networks; A spatiotemporal feature extraction module, which uses a temporal convolution kernel and a spatial convolution kernel to extract task-related spatiotemporal features from the EEG-fNIRS data after information intensive processing to identify changes in attention; The classification result acquisition module outputs the recognition result at the corresponding moment and determines the attention change.

2. A high-density EEG-fNIRS attention monitoring system based on AFHRF according to claim 1, characterized in that: The information intensive processing module specifically includes: an AFHRF generation module and an EEG-fNIRS fusion feature extraction module. After the EEG-fNIRS data is based on the AFHRF function generated by the AFHRF generation module, the cross-modal correlation features of the two are extracted by the EEG-fNIRS fusion feature extraction module.

3. A high-density EEG-fNIRS attention monitoring system based on AFHRF according to claim 2, characterized in that: The AFHRF generation module specifically includes: A sample data acquisition and preprocessing module, in which the subject completes a neural activation task, synchronously acquires and records the subject's EEG and fNIRS sample data, and preprocesses the sample data; The feature extraction network training module uses the synchronously acquired fNIRS data as labels to train the EEG feature extraction feature network; The AFHRF calculation module generates simulated fNIRS data through the feature extraction network and calculates the adaptive hemodynamic response function through deconvolution.

4. A high-density EEG-fNIRS attention monitoring system based on AFHRF according to claim 3, characterized in that: The feature extraction network training module uses a spatiotemporal feature convolutional network to extract EEG features, reduces the dimension to the same as fNIRS data through a fully connected layer, and calculates the loss function through a mean square error.

5. The high-density EEG-fNIRS attention monitoring system based on AFHRF according to claim 3, characterized in that: The sample data is divided into a training set and a test set according to a set ratio. The feature extraction network training module uses the fNIRS data of the training set collected synchronously as a label to train the feature extraction network to extract features of the EEG data of the corresponding channel in the training set.

6. The high-density EEG-fNIRS attention monitoring system based on AFHRF according to claim 4, characterized in that: The AFHRF calculation module imports the EEG data in the test set into the feature extraction network to generate simulated fNIRS data, uses the simulated fNIRS data as the convolved data, and the corresponding channel real EEG data as the event matrix, and deconvolutes to calculate the adaptive hemodynamic response function.

7. The high-density EEG-fNIRS attention monitoring system based on AFHRF according to claim 5, characterized in that: The AFHRF generation module also includes an AFHRF accuracy testing module, which, after importing the EEG data in the test set into the feature extraction network to generate simulated fNIRS data, also includes comparing the simulated fNIRS data with real fNIRS data to test the accuracy of the simulated fNIRS data.

8. The high-density EEG-fNIRS attention monitoring system based on AFHRF according to claim 4, characterized in that: The temporal convolution kernel of the spatiotemporal feature convolutional network is mathematically expressed as follows: X'=Conv1d(X,W t )+b t Where: X is the input EEG data, with a shape of (N, C, T), N is the number of samples, C is the number of channels or electrodes, and T is the time step; Wt is the temporal convolution kernel, with a shape of (Cout, 1, 1), where Cout is the number of output channels; bt is the bias term, with a shape of (Cout, 1, 1); X′ is the output after convolution, with a shape of (N, Cout, T).

9. The high-density EEG-fNIRS attention monitoring system based on AFHRF according to claim 1, characterized in that: The spatial convolution kernel of the convolutional network of the spatiotemporal feature is mathematically expressed as follows: Y=Conv2d(X,W s )+b s Where: Ws is the spatial convolution kernel, shape is (Tout, Cout, 1, 1), Tout is the time step, Cout is the number of output channels; bs is the bias term, shape is (Tout, Cout); Y is the output after convolution, shape is (N, Tout, Cout).

10. The high-density EEG-fNIRS attention monitoring system based on AFHRF according to claim 1, characterized in that: The classification result acquisition module is composed of a Focallose loss function and a 3-layer multi-layer perceptron, and the Focallose loss function is directly connected to the spatiotemporal feature extraction module. During training, each training is completed first with the training of the spatiotemporal feature extraction module, and then with the training of the information intensive processing module. The two trainings are isolated and trained alternately. The attention result output by the classification result acquisition module includes both the cognitive event category and the probability of the corresponding activation event.

Images