An electroencephalogram cognitive load evaluation method and system based on a multi-feature domain attention network

Abstract

The application belongs to the technical field of electroencephalogram data processing, and discloses an electroencephalogram cognitive load evaluation method and system based on a multi-feature domain attention network. The method first constructs a three-dimensional feature input according to the neural response of brain processing tasks, then constructs a multi-angle and diversity feature candidate set through multi-scale convolution, and combines the low-layer attention features with the high-layer attention features in a leapfrog manner to learn the multi-angle and diversity attention features of different sizes, so as to facilitate the high-level feature combination of the network and realize the effective evaluation of three load levels, i.e. low, medium and high. It is found through experimental results that the network constructed by the application has the highest classification accuracy compared with other four popular networks, thereby effectively improving the evaluation performance of electroencephalogram cognitive load.

Description

Technical Field

[0001] This invention relates to the field of EEG data processing technology, and in particular to a method and system for assessing EEG cognitive load based on a multi-feature domain attention network. Background Technology

[0002] Electroencephalogram (EEG) signals, as a neural response, can promptly reflect an operator's cognitive state during task execution. The level of cognitive load directly determines the operator's initiative and productivity. Therefore, assessing operator cognitive load to avoid overload and underload is crucial for safe production. However, the weakness and non-stationarity of EEG signals limit the performance of cognitive load assessment. Most current mainstream feature extraction methods employ single fusion approaches, failing to extract high-level features. Therefore, improving feature fusion and representation capabilities from multiple perspectives is significant for meeting the short-term needs and generalization of cognitive load assessment. Summary of the Invention

[0003] This invention addresses the problem that most current mainstream feature extraction methods employ single fusion approaches, failing to extract high-level features. It proposes a method and system for EEG cognitive load assessment based on a multi-feature-domain attention network. Based on the spatiotemporal structure of multi-channel EEG signals, a multi-feature-domain attention network for cognitive load assessment is proposed. Following the neural mechanisms of cognitive tasks, EEG data is integrated into a three-dimensional feature input. Local and global features are extracted through multi-scale convolution and a cross-connected Swin-Transformer structure. Finally, dimensionality reduction is achieved through average pooling, and classification is performed using fully connected layers.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] This invention proposes a method for assessing EEG cognitive load based on a multi-feature-domain attention network. The multi-feature-domain attention network includes multi-scale convolutions, a cross-connected Swin-Transformer structure, average pooling layers, and fully connected layers. The method comprises:

[0006] Step 1: Input the raw EEG data into three dimensions in the spatial, temporal, and frequency domains according to the brain's task processing mechanism;

[0007] Step 2: Extract local features from different receptive fields through multi-scale convolution to increase the richness of the feature candidate set;

[0008] Step 3: By using a cross-connected Swin-Transformer structure, low-level attention features are combined with high-level attention features in a cross-connected manner to learn diverse attention features of different sizes;

[0009] Step 4: Dimensionality reduction is performed using an average pooling layer, and classification is performed using a fully connected layer;

[0010] Furthermore, prior to step 1, the following steps are also included:

[0011] The simulation task was designed as follows: images with masking rates of 45%–55%, 65%–75%, and 85%–95% were selected from the publicly available SynISAR dataset as low, medium, and high workload levels. The rotation angle of the images of different targets under the same masking rate was kept consistent. Six aircraft models were selected for the formal experiment, and 45 images were selected for each model under a single workload level, for a total of 45×6×3 masked images. EEG signals were acquired using the g.HIamp EEG acquisition system.

[0012] Further, step 1 includes:

[0013] Time-frequency plots of neural responses in each lead within 1 second before key presses in a simulated task were plotted. First, short-time Fourier transform was used to convert the EEG data from different electrode channels into time-frequency plots. The spectrogram was obtained using MATLAB's built-in spectrum function, and a single-channel time-frequency plot of size 70×81 was obtained by setting parameters of Hamming window length 16, overlap window length 13, and Fourier transform points 256, representing the frequency domain energy from 0 to 60 Hz and the time domain information from 0 to 1000 ms, respectively. The Hamming window was used to reduce spectral leakage and maintain good frequency resolution. Then, the time-frequency plots of all channels were superimposed to construct a 62×70×81 three-dimensional input tensor.

[0014] Furthermore, in step 2, a multi-scale convolutional module is used for local feature extraction, and the various convolutional modules are concatenated in parallel to increase the richness of the feature candidate set. The multi-scale convolutional processing flow includes 1×1 convolutional layers, 3×3 convolutional layers, 5×5 convolutional layers, and max pooling layers, with each branch containing only one convolutional kernel of a unique size. To ensure the consistency of feature sizes after convolution at different scales, dimensionality reduction is first performed using 1×1 convolutional layers to reduce network parameters and integrate local correlations, and then the corresponding three-dimensional tensor is filled according to the size of the convolutional kernel.

[0015] Further, step 3 includes:

[0016] A cross-layered Swin-Transformer module is constructed to extract hierarchical global attention features. It consists of three stages, each with two Swin-Transformer blocks. In each stage, the input feature matrix is ​​first partitioned into non-overlapping patches using a patch partitioning layer, and linear embedding is applied to linearly transform the data in each channel. For a given input size of C×H×W, the output feature sizes of the three stages are C×H / 4×W / 4, 2C×H / 8×W / 8, and 4C×H / 16×W / 16, respectively. Dense connections are established between preceding Swin-Transformer blocks and subsequent layers using skip connections. To ensure size consistency when concatenating feature maps from different layers, appropriate convolution sizes are selected for each layer before performing convolution operations on the preceding layer's feature maps, followed by batch normalization of the feature layers before concatenation to avoid differences in data magnitude.

[0017] Another aspect of this invention proposes an EEG cognitive load assessment system based on a multi-feature-domain attention network. The multi-feature-domain attention network includes multi-scale convolutions, a cross-connected Swin-Transformer structure, average pooling layers, and fully connected layers. The system comprises:

[0018] The three-dimensional feature matrix construction module is used to input three-dimensional features of raw EEG data in the spatial, temporal, and frequency domains according to the brain's task processing mechanism.

[0019] The multi-scale convolution module is used to extract local features in different receptive fields through multi-scale convolution, thereby increasing the richness of the feature candidate set.

[0020] The cross-connected Swin-Transformer module is used to combine low-level attention features with high-level attention features across a cross-connected Swin-Transformer structure in order to learn diverse attention features of different sizes.

[0021] The classification module is used for dimensionality reduction through average pooling and classification using fully connected layers.

[0022] Furthermore, it also includes:

[0023] The simulation task design module is used to design simulation tasks, including: selecting images with masking rates of 45%–55%, 65%–75%, and 85%–95% from the publicly available SynISAR dataset as low, medium, and high workload levels. The rotation angle of images of different targets under the same masking rate is kept consistent. Six aircraft models are selected for formal experiments, and 45 images are selected for each model under a single workload level, for a total of 45×6×3 masking images. EEG signals are acquired using the g.HIamp EEG acquisition system.

[0024] Furthermore, the three-dimensional feature matrix construction module is specifically used for:

[0025] To generate time-frequency maps of neural responses in each lead within 1 second before image discrimination button presses in a simulated task, a short-time Fourier transform was first used to convert the EEG data of different electrode channels into time-frequency maps. The spectrogram was obtained using the built-in spectrum function of MATLAB. By setting parameters of Hamming window length 16, overlap window length 13, and Fourier transform points of 256, a single-channel time-frequency map of size 70×81 was obtained, representing the frequency domain energy of 0–60Hz and the time domain information of 0–1000ms, respectively. Then, the time-frequency maps of all channels were superimposed to construct a 62×70×81 three-dimensional input tensor.

[0026] Furthermore, in the multi-scale convolution module, multi-scale convolution is used for local feature extraction, and the various convolutions are concatenated in parallel. The multi-scale convolution processing flow includes 1×1 convolutional layers, 3×3 convolutional layers, 5×5 convolutional layers and max pooling layers, and different branches contain only one convolutional kernel with a unique size.

[0027] Furthermore, the cross-connected Swin-Transformer module is specifically used for:

[0028] A cross-connected Swin-Transformer structure is constructed to extract hierarchical global attention features. It consists of three stages, each with two Swin-Transformer blocks. In each stage, the input feature matrix is ​​first divided into non-overlapping patches through a patch partitioning layer, and the data of each channel is linearly transformed through linear embedding. For a given input size of C×H×W, the output feature sizes of the three stages are C×H / 4×W / 4, 2C×H / 8×W / 8, and 4C×H / 16×W / 16, respectively. Short-circuit connections are used to establish connections.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] This invention addresses the problem that most current mainstream feature extraction methods employ single fusion approaches, failing to extract high-level features. It proposes a method and system for EEG cognitive load assessment based on a multi-feature-domain attention network. Based on the spatiotemporal structure of multi-channel EEG signals, a multi-feature-domain attention network for cognitive load assessment is proposed. Following the neural mechanisms of cognitive tasks, EEG data is integrated into a three-dimensional feature input. Local and global features are extracted through multi-scale convolution and a cross-connected Swin-Transformer structure. Finally, dimensionality reduction is achieved through average pooling, and classification is performed using fully connected layers.

[0031] This invention combines low-level attention features with high-level attention features in a leapfrog manner, enabling the learning of diverse attention features of different sizes. This facilitates the combination of high-level features of the network and allows for effective evaluation of low, medium, and high load levels.

[0032] Experimental results show that the network constructed in this invention achieves the highest classification accuracy compared to four other popular networks, thereby effectively improving the performance of EEG cognitive load assessment. Attached Figure Description

[0033] Figure 1 This is a basic flowchart of an EEG cognitive load assessment method based on a multi-feature domain attention network according to an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the overall framework of the multi-feature domain attention network according to an embodiment of the present invention;

[0035] Figure 3 This is a simulation task experimental paradigm for embodiments of the present invention;

[0036] Figure 4 This is the data processing flow of an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of the cross-connected Swin-Transformer architecture according to an embodiment of the present invention;

[0038] Figure 6 Visual analysis of features in embodiments of the present invention;

[0039] Figure 7 This is a schematic diagram of the architecture of an EEG cognitive load assessment system based on a multi-feature domain attention network, according to an embodiment of the present invention. Detailed Implementation

[0040] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0041] like Figure 1As shown, a brainwave cognitive load assessment method based on a multi-feature domain attention network includes:

[0042] Step S101: Input the raw EEG data into three-dimensional features in the spatial, temporal, and frequency domains according to the brain's task processing mechanism;

[0043] Step S102: Extract local features from different receptive fields through multi-scale convolution to increase the richness of the feature candidate set;

[0044] Step S103: By using a cross-connected Swin-Transformer structure, low-level attention features are combined with high-level attention features in a cross-connected manner to learn diverse attention features of different sizes;

[0045] Step S104: Dimensionality reduction is performed using an average pooling layer, and classification is performed using a fully connected layer.

[0046] Figure 2 The diagram shows the overall architecture of a multi-feature domain attention network model, a serial fusion model based on CNN and Swin-Transformer. CNN is incorporated due to its excellent local feature representation properties, while Swin-Transformer is included because of its hierarchical representation and intra-window interaction characteristics. The model mainly includes multi-scale convolutions, a cross-connected Swin-Transformer structure, average pooling layers, and fully connected layers.

[0047] Furthermore, prior to step S101, the method also includes: simulation task design.

[0048] This embodiment designs a simulated image recognition task, which includes three workload levels, such as... Figure 3 As shown. The original experimental stimulus materials were derived from the publicly available SynISAR dataset, containing images of seven aircraft models. To enhance the realism of the simulation, further operations such as image magnification, pseudo-color filling, image masking, and rotation were performed on the images. Images with masking rates of 45%–55%, 65%–75%, and 85%–95% were selected from the simulated images as low, medium, and high workload levels. To ensure consistency in task difficulty, the image rotation angles for different targets at the same masking rate remained consistent. A total of six aircraft models were selected for the formal experiment, with 45 images selected for each model at a single workload level, resulting in a total of 45×6×3 masked images. Figure 3 As shown, the simulation experiment includes three sets of image recognition tasks. Each set of tasks is for a workload level and includes the discrimination of 45×6 images of 6 targets.

[0049] The simulated image recognition task was conducted in the laboratory. During the task, the g.HIamp EEG acquisition system provided by g.tec of Austria was used to acquire EEG signals, which includes 62 valid EEG signal channels. The electrode placement of the EEG system conformed to the international 10-20 standard system, the online sampling frequency was 512Hz, the bandpass filter was 0.01~00Hz, and the notch filter frequency was 50Hz.

[0050] Further, step S101 includes:

[0051] like Figure 4 As shown, a bandpass filter of 0.1–60 Hz was used offline to filter out slow drift and high-frequency noise during the preprocessing stage. The bandpass filter consisted of a low-pass filter and a high-pass Chebyshev filter. The low-pass filter (order, 3; stopband start frequency, 50 Hz; stopband cutoff frequency, 60 Hz; passband attenuation, 0.5 dB; stopband attenuation, 5 dB) and the high-pass filter (order, 1; stopband start frequency, 0.01 Hz; stopband cutoff frequency, 1 Hz; passband attenuation, 1 dB; stopband attenuation, 10 dB) were obtained using built-in MATLAB functions. The mean-referenced method was used for re-reference, which is considered to have higher accuracy in traceability and brain network analysis. To reduce baseline differences caused by data instability, the EEG signal 200 ms before stimulus presentation was used as the baseline. Data from damaged leads were replaced with the mean of adjacent leads. Independent component analysis was used to remove electrooculography artifacts from the signal. The preprocessed EEG data was downsampled to 256 Hz to reduce data processing and computation. After the stimulus is presented, the EEG data of the previous 1 second is selected as the single sample data, and the workload level of the task is used as the sample label. The final sample data size is 25×3×6×45 (person-times×workload level×number of targets×number of images per target).

[0052] To fully utilize the useful information in each dimension of EEG signals and present a more comprehensive and richer set of EEG features, this paper constructs multi-domain feature representations in the spatial, frequency, and temporal domains, reflecting neural activity in cognitive tasks from multiple perspectives. For example... Figure 4As shown, in the feature extraction stage, the time-frequency map of EEG data from different electrode channels is first converted using short-time Fourier transform. The spectrogram is obtained using the built-in spectrum function of MATLAB, and a single-channel time-frequency map of size 70×81 is obtained by setting parameters of Hamming window length 16, overlap window length 13, and Fourier transform points 256, representing the frequency domain energy from 0 to 60 Hz and the time domain information from 0 to 1000 ms, respectively. The Hamming window is used to reduce spectral leakage and maintain good frequency resolution. Then, the time-frequency maps of all channels are superimposed to construct a 62×70×81 three-dimensional input tensor to represent the spatial, frequency, and temporal characteristics of EEG, which is beneficial for multi-angle feature fusion in network training.

[0053] When conducting cognitive load assessment, this invention attempts to construct a multi-angle representation of EEG signals, effectively integrating spatial, temporal, and frequency domain information from EEG features to effectively learn the connections between dynamic changes in the brain.

[0054] Further, step S102 includes:

[0055] To increase the receptive field of the network, this embodiment employs multi-scale convolution for feature extraction, concatenating multiple convolutions in parallel to form the first half of the overall network architecture. The multi-scale convolution processing flow includes 1×1 convolutional layers, 3×3 convolutional layers, 5×5 convolutional layers, and max-pooling layers, with each branch containing only one convolutional kernel of a unique size. To ensure consistency in feature size after convolution at different scales, dimensionality reduction is first performed using 1×1 convolutional layers to reduce network parameters and integrate local correlations. Then, the corresponding 3D tensor is filled according to the size of the convolutional kernel. Multi-branch convolution, as an important structure for multi-scale feature extraction, is beneficial for capturing EEG features from macroscopic to detailed perspectives and extracting diverse complementary information. The output of this step contains the comprehensive features extracted by the multiple convolutional kernels and transmits the 3D information to the next stage.

[0056] Further, step S103 includes:

[0057] To increase global feature information in the network while minimizing computational cost, the Swin-Transformer model uses a moving windowing scheme to limit self-attention computation to non-overlapping local windows, and a hierarchical global attention mechanism leads to higher network efficiency. For example... Figure 5As shown, this embodiment constructs a skip-connected Swin-Transformer structure to extract hierarchical global attention features. It consists of three stages, each with two Swin-Transformer blocks. In each stage, the input feature matrix is ​​first divided into non-overlapping patches using a patch partitioning layer, and then linearly transformed for each channel using linear embedding. For a given input size of C×H×W, the output feature sizes for the three stages are C×H / 4×W / 4, 2C×H / 8×W / 8, and 4C×H / 16×W / 16, respectively. These are then processed by the corresponding Swin-Transformer blocks. To comprehensively utilize the shallow, low-complexity attention features and improve feature utilization, a skip connection approach is used to establish dense connections between the preceding Swin-Transformer blocks and the following layers. To ensure consistent size when concatenating feature maps from different layers, appropriate convolutional sizes are selected for each layer's feature maps. After batch normalization of the feature maps, the concatenation operation is performed to avoid differences in data magnitude. Finally, a 1×1 convolutional layer is used to ensure consistency between the output size of the merged feature layer and the current feature layer.

[0058] To verify the effectiveness of the present invention, the following experiment was conducted:

[0059] (1) Comparison of multi-feature domain attention networks with other baseline methods:

[0060] To verify the effectiveness of the multi-feature domain attention network model, its performance was compared with four commonly used mainstream network models: CNN, CNN-LSTM, Transformer, and Swin-Transformer. The comparison results are shown in Table 1. CNN is a feedforward neural network that exhibits high recognition performance in EEG signal classification. CNN-LSTM is a hybrid model of CNN and LSTM, utilizing the representational power of convolutional layers and the ability of LSTM to capture temporal correlations to extract EEG features. Transformer is a network model based on an encoder-decoder structure, extracting global EEG features through a unique attention mechanism. Swin-Transformer is a novel visual Transformer proposed by Ze Liu et al., which enhances the performance of the Transformer network structure through hierarchy, locality, and translation invariance. The results show that the multi-feature domain attention network model achieves the best performance in three workload evaluations compared to other mainstream networks.

[0061] Table 1 Comparison of the methods

[0062]

[0063] The limitation of other networks is their model homogeneity, neglecting the correlations between features across different dimensions and regions. Since brain regions exist in non-Euclidean space, flattening EEG channels distributed on an irregular grid into a two-dimensional representation with a regular grid cannot accurately reflect the spatial relationships of signals. In contrast, the multi-feature domain attention network model used in this paper adds attention features at different scales between brain regions through a hierarchical structure, and the windowing operation also brings higher efficiency to the network itself. The use of skip connections introduces new interactions to the spatial dimension of the attention mechanism. Therefore, the proposed method outperforms other baseline methods, achieving 11.29% and 24.72% higher accuracy than Swin-Transformer and CNN models, respectively, and 18.87% and 20.96% higher accuracy than Transformer and CNN-LSTM models, respectively.

[0064] (2) Ablation experiment of multi-feature domain attention network model:

[0065] To verify the contribution of each module to the model performance, ablation experiments were conducted on the dataset created in this invention, and the results are shown in Table 2. First, ablation experiments were performed on the convolution kernel combination strategy of the multi-scale convolution module, exploring the effects of single-scale convolution kernels (SSC), dual-scale convolution kernels (DSC), and multi-scale convolution kernels (MSC) on the model. The results show that among single-scale convolution kernels, the 1×1 convolution kernel outperforms the 3×3 and 5×5 convolution kernels. Among dual-scale convolution kernels, the combination strategy of 1×1 and 3×3 convolution kernels outperforms the combination strategy of 1×1 and 5×5 convolution kernels and the combination strategy of 3×3 and 5×5 convolution kernels. Compared to the above two strategies, the multi-scale convolution kernel combination strategy has the best performance, achieving a workload evaluation accuracy of 87.4 ± 0.78.

[0066] Table 2 Ablation Experiment

[0067]

[0068] To test the impact of "short-circuit connections," performance tests were conducted separately on the straddle-type Swin-Transformer structure. Compared to the convolutional module, the performance improvement of this structure was relatively small. For the entire module, the fusion strategy of local and global features can more accurately reflect the differences between different workload levels, demonstrating the superiority of the proposed model in terms of performance improvement.

[0069] (3) Model visualization analysis:

[0070] To further analyze the extraction effect of each module on cognitive load features, we conducted a visualization analysis of the time-frequency features of different layers in the model. The output of each layer was normalized to the attention intensity of the feature map by z-score and then the corresponding feature heatmap was drawn. Figure 6 The visualization results of the feature extraction process are shown. For multi-scale convolutional layers, we can see that convolutional kernels of different scales focus on capturing low-frequency signal features, while for mid- and high-frequency signals, the attention regions of different convolutional kernels exhibit diverse and complementary results. It is evident that the operation of multi-scale parallel convolution effectively preserves the original EEG features and captures the diversity and complementarity of EEG features. The attention regions obtained through the cross-connected Swin-Transformer structure of this invention show a trend of approaching the center from the top and bottom. Compared to the characteristic of convolutional modules focusing on local features, this invention demonstrates the advantage of integrating global contextual features.

[0071] Based on the above embodiments, such as Figure 7 As shown, this invention also proposes an EEG cognitive load assessment system based on a multi-feature-domain attention network. The multi-feature-domain attention network includes multi-scale convolutions, a cross-connected Swin-Transformer structure, average pooling layers, and fully connected layers. The system comprises:

[0072] The three-dimensional feature matrix construction module is used to input three-dimensional features of raw EEG data in the spatial, temporal, and frequency domains according to the brain's task processing mechanism.

[0073] The multi-scale convolution module is used to extract local features in different receptive fields through multi-scale convolution, thereby increasing the richness of the feature candidate set.

[0074] The cross-connected Swin-Transformer module is used to combine low-level attention features with high-level attention features across a cross-connected Swin-Transformer structure in order to learn diverse attention features of different sizes.

[0075] The classification module is used for dimensionality reduction through average pooling and classification using fully connected layers.

[0076] Furthermore, it also includes:

[0077] The simulation task design module is used to design simulation tasks, including: selecting images with masking rates of 45%–55%, 65%–75%, and 85%–95% from the publicly available SynISAR dataset as low, medium, and high workload levels. The rotation angle of images of different targets under the same masking rate is kept consistent. Six aircraft models are selected for formal experiments, and 45 images are selected for each model under a single workload level, for a total of 45×6×3 masking images. EEG signals are acquired using the g.HIamp EEG acquisition system.

[0078] Furthermore, the three-dimensional feature matrix construction module is specifically used for:

[0079] To generate time-frequency maps of neural responses in each lead within 1 second before image discrimination button presses in a simulated task, a short-time Fourier transform was first used to convert the EEG data of different electrode channels into time-frequency maps. The spectrogram was obtained using the built-in spectrum function of MATLAB. By setting parameters of Hamming window length 16, overlap window length 13, and Fourier transform points of 256, a single-channel time-frequency map of size 70×81 was obtained, representing the frequency domain energy of 0–60Hz and the time domain information of 0–1000ms, respectively. Then, the time-frequency maps of all channels were superimposed to construct a 62×70×81 three-dimensional input tensor.

[0080] Furthermore, in the multi-scale convolution module, multi-scale convolution is used for local feature extraction, and the various convolutions are concatenated in parallel. The multi-scale convolution processing flow includes 1×1 convolutional layers, 3×3 convolutional layers, 5×5 convolutional layers and max pooling layers, and different branches contain only one convolutional kernel with a unique size.

[0081] Furthermore, the cross-connected Swin-Transformer module is specifically used for:

[0082] A cross-connected Swin-Transformer structure is constructed to extract hierarchical global attention features. It consists of three stages, each with two Swin-Transformer blocks. In each stage, the input feature matrix is ​​first divided into non-overlapping patches through a patch partitioning layer, and the data of each channel is linearly transformed through linear embedding. For a given input size of C×H×W, the output feature sizes of the three stages are C×H / 4×W / 4, 2C×H / 8×W / 8, and 4C×H / 16×W / 16, respectively. Short-circuit connections are used to establish connections.

[0083] In summary, this invention addresses the problem that most mainstream feature extraction methods employ single fusion approaches, failing to extract high-level features. It proposes a multi-feature-domain attention network-based EEG cognitive load assessment method and system. Based on the spatiotemporal structure of multi-channel EEG signals, a multi-feature-domain attention network for cognitive load assessment is proposed. Following the neural mechanisms of cognitive tasks, EEG data is integrated into three-dimensional feature inputs. Local and global features are extracted through multi-scale convolution and a cross-connected Swing-Transformer structure. Finally, dimensionality reduction is achieved through average pooling, and classification is performed using fully connected layers. This invention learns diverse attention features of different sizes by combining low-level attention features with high-level attention features in a cross-connected manner, facilitating high-level feature combinations and enabling effective assessment of low, medium, and high load levels. Experimental results show that the network constructed in this invention achieves the highest classification accuracy compared to four other popular networks, thus effectively improving the performance of EEG cognitive load assessment.

[0084] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for assessing cognitive load based on multi-feature domain attention networks, characterized in that, The multi-feature domain attention network includes multi-scale convolutions, a cross-connected Swin-Transformer structure, average pooling layers, and fully connected layers. The method includes: Step 1: Input the raw EEG data into three dimensions in the spatial, temporal, and frequency domains according to the brain's task processing mechanism; Step 2: Extract local features from different receptive fields through multi-scale convolution to increase the richness of the feature candidate set; In Step 2, multi-scale convolution is used for local feature extraction, and the various convolutions are concatenated in parallel. The multi-scale convolution processing flow includes 1×1 convolutional layers, 3×3 convolutional layers, 5×5 convolutional layers and max pooling layers, and different branches contain only one convolutional kernel with a unique size; Step 3: Combine low-level attention features with high-level attention features in a cross-connected Swin-Transformer structure to learn diverse attention features of different sizes; Step 3 includes: constructing a cross-connected Swin-Transformer structure to extract hierarchical global attention features, which is divided into 3 stages, each stage has 2 Swin-Transformer blocks. In each stage, the input feature matrix is ​​first divided into non-overlapping patches through a patch partitioning layer, and the data of each channel is linearly transformed through linear embedding. For a given input size of C×H×W, the output feature sizes of the three stages are C×H / 4×W / 4, 2C×H / 8×W / 8 and 4C×H / 16×W / 16, respectively, and the connections are established using a short-circuit connection method; Step 4: Dimensionality reduction is performed using an average pooling layer, and classification is performed using a fully connected layer.

2. The EEG cognitive load assessment method based on a multi-feature domain attention network according to claim 1, characterized in that, Before step 1, the following is also included: The simulation task was designed as follows: images with masking rates of 45%~55%, 65%~75%, and 85%~95% were selected from the publicly available SynISAR dataset as low, medium, and high workload levels. The rotation angle of the images of different targets under the same masking rate was kept consistent. Six aircraft models were selected for formal experiments, and 45 images were selected for each model under a single workload level, for a total of 45×6×3 masked images. EEG signals were acquired using the g.HIamp EEG acquisition system.

3. The EEG cognitive load assessment method based on a multi-feature domain attention network according to claim 2, characterized in that, Step 1 includes: To generate time-frequency plots of neural responses in each lead within 1 second before image discrimination button presses in a simulated task, the time-frequency plots of EEG data from different electrode channels were first converted using short-time Fourier transform. The spectrograms were obtained using the built-in spectrum function in MATLAB. By setting parameters of Hamming window length 16, overlap window length 13, and Fourier transform points of 256, a single-channel time-frequency plot of size 70×81 was obtained, representing the frequency domain energy from 0 to 60 Hz and the time domain information from 0 to 1000 ms, respectively. Then, the time-frequency plots of all channels were superimposed to construct a 62×70×81 three-dimensional input tensor.

4. A brainwave cognitive load assessment system based on a multi-feature domain attention network, characterized in that, The multi-feature domain attention network includes multi-scale convolutions, a cross-connected Swin-Transformer structure, average pooling layers, and fully connected layers. The system comprises: The three-dimensional feature matrix construction module is used to input three-dimensional features of raw EEG data in the spatial, temporal, and frequency domains according to the brain's task processing mechanism. The multi-scale convolution module is used to extract local features in different receptive fields through multi-scale convolution, thereby increasing the richness of the feature candidate set. In the multi-scale convolution module, multi-scale convolution is used for local feature extraction, and the various convolutions are concatenated in parallel. The multi-scale convolution processing flow includes 1×1 convolutional layers, 3×3 convolutional layers, 5×5 convolutional layers and max pooling layers. Different branches contain only one convolutional kernel with a unique size. The cross-connected Swin-Transformer module is used to combine low-level attention features with high-level attention features across a cross-connected Swin-Transformer structure to learn diverse attention features of different sizes. Specifically, the cross-connected Swin-Transformer module is used to construct a cross-connected Swin-Transformer structure to extract hierarchical global attention features. It is divided into three stages, each with two Swin-Transformer blocks. In each stage, the input feature matrix is ​​first divided into non-overlapping patches through a patch partitioning layer, and the data of each channel is linearly transformed through linear embedding. For a given input size of C×H×W, the output feature sizes of the three stages are C×H / 4×W / 4, 2C×H / 8×W / 8, and 4C×H / 16×W / 16, respectively. Connections are established using a short-circuit connection method. The classification module is used for dimensionality reduction through average pooling and classification using fully connected layers.

5. The EEG cognitive load assessment system based on a multi-feature domain attention network according to claim 4, characterized in that, Also includes: The simulation task design module is used to design simulation tasks, including: selecting images with masking rates of 45%~55%, 65%~75%, and 85%~95% from the publicly available SynISAR dataset as low, medium, and high workload levels. The rotation angle of images of different targets under the same masking rate is kept consistent. Six aircraft models are selected for formal experiments, and 45 images are selected for each model under a single workload level, for a total of 45×6×3 masking images. EEG signals are acquired using the g.HIamp EEG acquisition system.

6. The EEG cognitive load assessment system based on a multi-feature domain attention network according to claim 5, characterized in that, The three-dimensional feature matrix construction module is specifically used for: To generate time-frequency plots of neural responses in each lead within 1 second before image discrimination button presses in a simulated task, the time-frequency plots of EEG data from different electrode channels were first converted using short-time Fourier transform. The spectrograms were obtained using the built-in spectrum function in MATLAB. By setting parameters of Hamming window length 16, overlap window length 13, and Fourier transform points of 256, a single-channel time-frequency plot of size 70×81 was obtained, representing the frequency domain energy from 0 to 60 Hz and the time domain information from 0 to 1000 ms, respectively. Then, the time-frequency plots of all channels were superimposed to construct a 62×70×81 three-dimensional input tensor.

Images