EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of biological feature recognition, and discloses an EEG visual classification method based on TS-BiGRU and a dynamic channel attention mechanism, an electronic device and a storage medium, and the EEG visual classification method comprises the steps: obtaining initial EEG signal data, carrying out preprocessing, dividing the preprocessed data, constructing a TS-BiGRU module, and constructing a TS-BiGRU module; establishing a training set electroencephalogram signal data, inputting the training set electroencephalogram signal data into a TS-BiGRU module to obtain time features, constructing a CAKE module containing an EFCM masking module, and inputting the training set electroencephalogram signal data into the CAKE module to obtain spatial features; splicing the features to obtain fused features; the fused features are transmitted to a feedforward neural network and then enter a softmax layer for prediction after passing through the feedforward neural network; the technical problem that the classification performance is poor due to the fact that the multi-dimensional features contained in the EEG signals in the time domain, the space domain and the frequency domain cannot be fully mined in the prior art is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of biometric recognition, and relates to an EEG visual classification method, an electronic device and a storage medium based on TS-BiGRU and a dynamic channel attention mechanism, which are applicable to EEG visual classification, especially EEG signals of non-invasive electroencephalogram acquisition signals. Background Art

[0002] Electroencephalogram (EEG) signals are non-invasive electrophysiological signals that reflect brain nerve activities and are widely used due to their advantages such as high temporal resolution, portability, and low cost. However, due to the highly non-linear, non-stationary, and low signal-to-noise ratio characteristics of EEG signals, how to extract effective features from them and achieve efficient classification is a key issue in current research.

[0003] Traditional EEG signal processing methods are based on machine learning and manual feature engineering, and have achieved certain results. However, when dealing with visually evoked EEG signals, they face many challenges. For example, traditional machine learning methods such as SVM (Support Vector Machine) and LDA (Linear Discriminant Analysis) rely on artificial extraction of features such as time-domain statistics and band power, and their performance is limited by expert experience, making it difficult to capture the non-linear dynamic features of EEG signals. Time-domain analysis (such as peak detection) has poor recognition accuracy for the latency of transient components such as P300; frequency-domain methods (such as power spectrum estimation) are difficult to effectively separate the fundamental frequency and harmonic components of SSVEP signals; spatial feature extraction often ignores the electrode topology structure, resulting in the loss of brain region cooperation information. In deep learning methods, although the CNN (Convolutional Neural Network) architecture can extract time-frequency features when processing EEG signal data, it is difficult to model long-range time dependencies such as those shown by the millisecond-level dynamics of P300. Existing RNN (Recurrent Neural Network) architectures have insufficient utilization of their spatial features when processing EEG signal data. Existing hybrid neural network architectures such as the STFT-CNN architecture that combines the Short-Time Fourier Transform (STFT) with the convolutional neural network or the CWT-CNN architecture that combines the Continuous Wavelet Transform (CWT) with the convolutional neural network can capture the time and frequency features of EEG signals simultaneously, but have insufficient utilization of the spatial information of multi-channel EEG signals, and these spatial information are important for revealing the response differences of different brain regions to visual stimuli, often splitting the correlation of spatio-temporal-frequency features; and due to the low signal-to-noise ratio of EEG signals, existing methods perform poorly in dealing with noise interference or processing small-sample data, and the overfitting problem in small-sample scenarios has not been effectively solved; in summary, there are three limitations in the existing technology for processing visually evoked EEG signals: 1) The collaborative extraction ability of time-domain-frequency-domain-spatial-domain features is significantly insufficient, and an effective cross-dimensional feature fusion mechanism has not been established; 2) The existing methods lack depth in the characterization of features in each dimension, especially in the ability to capture the dynamic evolution process in the time domain, and the spatial feature extraction method lacks flexibility; 3) The split processing of spatio-temporal-frequency features results in the inability to fully integrate the multi-dimensional discriminant information contained in EEG signals, restricting the improvement of classification performance.

[0004] Based on the limitations existing in the processing of visually evoked EEG signals, 2) the Chinese patent with the publication number CN113974658A extracts the features of VEP and N400 components through the EEG time-sharing spectrum Riemann method, and combines semantic judgment and visual observation to solve the problem of low accuracy in semantic visual image classification in BCI (Brain-Computer Interface) technology; Based on the limitations existing in the processing of visually evoked EEG signals, 3) the Chinese patent with the publication number CN114176607A solves the problem that it is difficult to utilize the local features of EEG signals by introducing the EEGViT model in EEG signal classification and combining the EEG Transformer Encoder and the SequenceIn Time Transformer Encoder. However, these methods still have problems such as insufficient long-range time dependence modeling, low efficiency in utilizing frequency domain features, and limited ability to dynamically extract spatial features.

[0005] To address the above problems, the present invention proposes an EEG visual classification method based on TS-BiGRU and a dynamic channel attention mechanism. It accurately captures long-range time dependence relationships layer by layer through a time-selective bidirectional gated recurrent unit, enhances the response of specific frequency bands by combining a learnable band energy modulation module, and adaptively learns the importance weights of each channel using the dynamic channel attention mechanism. The EEG visual classification method based on TS-BiGRU and the dynamic channel attention mechanism proposed by the present invention provides a new method for research in fields such as brain-computer interfaces. Summary of the Invention

[0006] The present invention provides an EEG visual classification method, an electronic device, and a storage medium based on TS-BiGRU and a dynamic channel attention mechanism, which solve the technical problems of insufficient ability in comprehensively extracting time, space, and frequency features when processing visually evoked EEG signals in the prior art, and being unable to fully exploit the multi-dimensional features contained in EEG signals in time, space, and frequency domain; thereby resulting in poor classification performance.

[0007] To solve the above technical problems, the technical solution of the present invention is an EEG visual classification method based on TS-BiGRU and a dynamic channel attention mechanism, and the specific steps are as follows:

[0008] S1. Obtain the initial EEG signal data, preprocess the initial EEG signal data, and divide the preprocessed initial EEG signal data into a training set and a test set;

[0009] S2. Construct a TS-BiGRU module, input the training set EEG signal data into the TS-BiGRU module, perform time feature extraction, and obtain the time features of the training set EEG signal data;

[0010] S3. Define an EFCM masking module, which is used to sort the channel importance of the EEG signal data in the training set;

[0011] S4. Construct a CAKE module containing an EFCM masking module, and input the training set EEG signal data into the CAKE module to perform spatial feature extraction to obtain the spatial features of the training set EEG signal data;

[0012] The CAKE module is used to dynamically extract the spatial features of the training set EEG signal data;

[0013] The CAKE module includes a multi-head attention mechanism, an EFCM module, and a soft routing mechanism;

[0014] S5. Combine the temporal features obtained in S2 and the spatial features obtained in S4 to obtain fused features;

[0015] S6. The fused features are passed to the feedforward neural network to obtain the probability distribution of each category, and then the probability distribution is normalized using the softmax function to obtain the classification results of the EEG signal data of the training set.

[0016] Furthermore, the specific steps of S1 are as follows:

[0017] S101: Acquire initial EEG signal data, and filter the initial EEG signal data using a frequency band filter to obtain filtered initial EEG signal data;

[0018] S102: Processing the filtered initial EEG signal data using a standardized algorithm, and dividing the processed data into a training set and a test set.

[0019] Furthermore, the TS-BiGRU module constructed by S2 includes: a time-selective update gate z t The forward GRU network, containing the time-selective update gate z t The backward GRU network with time-selective reset gate r t The forward GRU network, with a time-selective reset gate r t The backward GRU network has the following function expressions:

[0020]

[0021] Among them: z is the update gate output, f is the forward direction, t is the time step, is the output value of the forward GRU update gate at the current time step, σ is the sigmoid function, is the weight matrix applied to the input of the current time step in the forward GRU update gate, h t-1is the hidden state at the previous moment, is the weight matrix applied to the hidden state at the previous moment in the forward GRU update gate, x t is the input at the current time step, is the bias of the forward GRU update gate, λ is the decay coefficient, 0 < λ < 1;

[0022]

[0023] where: b is backward, is the output value of the backward GRU update gate at the current time step, is the weight matrix applied to the input at the current time step in the backward GRU update gate, h t+1 is the hidden state at the next moment, is the weight matrix applied to the hidden state at the next moment in the backward GRU update gate, is the bias of the backward GRU update gate, γ is the amplification coefficient, γ > 1;

[0024]

[0025] where: r is the output of the reset gate, is the output value of the forward GRU reset gate at the current time step, is the weight matrix applied to the input at the current time step in the forward GRU reset gate, is the weight matrix applied to the hidden state at the previous moment in the forward GRU reset gate, is the bias of the forward GRU reset gate, α is the amplification coefficient, α > 1;

[0026]

[0027] where: is the output value of the backward GRU reset gate at the current time step, is the weight matrix applied to the input at the current time step in the backward GRU reset gate, is the weight matrix applied to the hidden state at the next moment in the backward GRU reset gate, is the bias of the backward GRU reset gate, β is the decay coefficient, 0 < β < 1.

[0028] Furthermore, in the TS-BiGRU module constructed in S2, the function expression for calculating the candidate hidden state is shown in Equation (5), the function expression for updating the hidden state h t is shown in Equation (6), the function expression for calculating the backward candidate hidden state is shown in Equation (7), and the function expression for updating the backward hidden state is shown in Equation (8):

[0029]

[0030] Wherein: is the candidate hidden state, tanh is the tanh function, W h , U h is the weight matrix, x t is the input at the current moment, r t is the reset gate, h t-1 is the hidden state at the previous moment, b h is the bias term, h t is the updated hidden state, z t is the update gate, ⊙ is the Hadamard product, is the reverse input at the current moment, is the reverse reset gate, is the reverse hidden state at the next moment, is the reverse update gate, is the reverse candidate hidden state.

[0031] Furthermore, the output function expression of the TS-BiGRU module constructed in S2 is as follows:

[0032] output,h n =TSBiGRU(e;θ) (9)

[0033] Wherein: e represents the input sequence, c is the number of channels, d is the feature dimension, is the set of real numbers, θ is the trainable parameter in the TS-BiGRU model, output is the output sequence of the TS-BiGRU model at each time step, h n is the hidden state of the TS-BiGRU model at the last time step, and BiGRU is the bidirectional gated recurrent unit.

[0034] Furthermore, the specific steps for using the EFCM masking module to sort the channel importance of the training set EEG signal data in S3 are as follows:

[0035] S301: Let the training set EEG signal data be X, and denote the training set EEG signal data as a three-dimensional tensor Wherein, B is the batch size, C is the number of channels, t is the time step, and let the number of key channels to be selected by the EFCM masking module be k;

[0036] S302: Calculate the attention weights of each EEG signal data sample in the training set respectively, where the attention weight of the p-th sample is w p,i , and the attention weight w p,i of the p-th sample has the following function expression:

[0037]

[0038] Where: X p,i is the time series of the i-th channel of the p-th sample, i is the channel index, p is the sample, w p,i is the weight corresponding to the i-th channel in the p-th sample, is the transpose of the time series of the i-th channel of the p-th sample, t is the time step;

[0039] S303: Use the softmax function to normalize the attention weights of each EEG signal data sample in the training set. For the attention weight w p,i of the p-th sample, its normalized attention weight The function expression is as follows:

[0040]

[0041] Where: is the weight value corresponding to the i-th channel of the p-th sample after normalization, w p,j is the original weight value corresponding to the j-th channel of the p-th sample, j is the index channel number, exp(.) is the natural exponential function;

[0042] S304: Normalize the attention weights of each EEG signal data sample in the training set, sort the normalized attention weights w according to the channels, and determine the retained channel indices according to the key channel number value k described in S301 to obtain its corresponding index set;

[0043] The normalized attention weight of the p-th sample in the training set corresponds to the sorted index set S P The function expression is as follows:

[0044] S p ={S P,1 ,…,S P,k} (12).

[0045] Furthermore, the specific steps of S4 are as follows:

[0046] S401: Input the training set EEG signal data into the CAKE module, perform matrix multiplication, scaling, and softmax operations on the training set EEG signal data in sequence to obtain the initial attention weights. Input the initial attention weights into the EFCM masking module in the CAKE module, calculate the attention weights corresponding to each path according to the channel differences masked by different paths, and sum the attention weights corresponding to multiple paths and the path factor r kMultiply to obtain path-enhanced attention weights;

[0047] S402: Multiply the path-enhanced attention weights by the value matrix V to obtain the output result of a single attention head. Concatenate the output results of all H attention heads to form a multi-head attention output matrix, and perform a linear projection process on the multi-head attention output matrix to obtain the spatial features of the training set EEG signal data.

[0048] Further, the specific steps of S401 are as follows:

[0049] Step S4011: Input each EEG signal data sample in the training set into the CAKE module, and use the CAKE module to perform a linear transformation on the channel features of each data sample to obtain a query matrix Q, a key matrix K, and a value matrix V. For the p-th sample, the query matrix Q p The functional expression of is shown in Equation (13), and the key matrix K p The functional expression of is shown in Equation (14), and the value matrix V p The functional expression of is shown in Equation (15):

[0050] Q p = X p W Q (13)

[0051] K p = X p W K (14)

[0052] V p = X p W V (15)

[0053] Where: Q p is the query matrix corresponding to the p-th sample, K p is the key matrix corresponding to the p-th sample, V p is the value matrix corresponding to the p-th sample, X p is the input data sample, W Q , W K , W V are trainable parameter matrices, d model = H × d head , H is the number of attention heads, d head is the dimension of each attention head, X p is the p-th sample data, C is the number of channels, d model is the feature dimension, is the set of real numbers;

[0054] Step S4012: Split the query matrix Q, key matrix K, and value matrix V corresponding to each EEG signal data sample in the training set into H attention heads respectively. For the query matrix Q corresponding to the p-th sample p , the key matrix K p , and the value matrix V p The split query matrix Q p As shown in Equation (16), the split key matrix K p As shown in Equation (17), the split value matrix V p As shown in Equation (18):

[0055] Q p,h = Q p [:,(h - 1)d head :hd head (16)

[0056] K p,h = K p [:,(h - 1)d head :hd head (17)

[0057] V p,h = V p [:,(h - 1)d head :hd head (18)

[0058] Where: Q p,h is the query matrix of the h-th attention head of the p-th sample, K p,h is the key matrix of the h-th attention head of the p-th sample, V p,h is the value matrix of the h-th attention head of the p-th sample, and h is the number of the attention head;

[0059] Step S4013: Input the attention score A corresponding to each EEG signal data sample in the training set into the EFCM masking module to obtain the attention score matrix of the corresponding path. For A corresponding to the p-th sample h , its functional expression is as shown in (19). Input the p-th sample into the EFCM module to obtain the attention score matrix of the path obtained by the EFCM masking module for the p-th sample at the h-th attention head as shown in Equation (20) p,h Where: A

[0060]

[0061] is the attention score corresponding to the p-th sample, softmax is the softmax function, Q p,h is the query matrix of the p-th sample at the h-th attention head, p,h and is the transpose of the key matrix of the p-th sample at the h-th attention head, where T represents transpose;

[0062] MA p,h = EFCM(A p,h ) (20)

[0063] where: MA p,h is the attention score matrix of multiple paths obtained by the EFCM masking module for the p-th sample at the h-th attention head;

[0064] Multiply the attention score matrix corresponding to multiple paths by the path factor r k to obtain the path-enhanced attention score matrix. Multiply the path-enhanced attention score matrix corresponding to each EEG signal data sample in the training set by the value matrix to obtain the weighted value matrix of each data sample. The weighted value matrix of the p-th sample corresponding to the h-th attention head is shown in Equation (21):

[0065] Z p,h = MA p,h r p,h,k V p,h (21)

[0066] where: Z p,h is the weighted value matrix of the p-th sample at the h-th attention head, r p,h,k is the path factor of the p-th sample at the h-th attention head, V p,h is the value matrix of the p-th sample at the h-th attention head, and k is the number of paths.

[0067] An electronic device includes a processor, a memory, and a communication bus. Among them, the processor and the memory complete communication with each other through the communication bus;

[0068] The memory is used to store computer programs;

[0069] The processor is used to implement the above method steps when executing the programs stored on the memory.

[0070] A computer storage medium stores a computer program therein, and the computer program realizes the above method steps when executed by a processor.

[0071] Advantages of the present invention:

[0072] First, aiming at the problem that existing methods are difficult to comprehensively consider the time, space, and frequency mode features of EEG signals, the present invention proposes an EEG visual classification method based on TS-BiGRU and a dynamic channel attention mechanism. By first using a band filter for band filtering and then adopting a parallel structure to extract spatio-temporal feature information simultaneously, on the one hand, the operation efficiency is improved, and on the other hand, the interference between spatio-temporal features is avoided, effectively solving the technical problem of difficult comprehensive utilization of multi-mode features and enhancing the expression ability of spatio-temporal features.

[0073] Secondly, to deeply mine the deep temporal feature information in EEG signals and solve the problem that existing methods are not deep enough in time feature extraction, the present invention uses TS-BiGRU in time feature extraction. TS-BiGRU can process EEG signals in both forward and reverse directions, comprehensively capture the variation law of signals in the time dimension, and effectively extract deep temporal features.

[0074] Thirdly, due to the problems that existing methods do not consider channel redundancy and do not use a masking mechanism to shield redundant channels when processing EEG spatial features, the present invention designs a dynamic channel attention mechanism (CAscaded elite-K dynamic channel attEntion, CAKE). Based on the masking mechanism, this mechanism can dynamically determine the masked channels according to the input data, eliminate the influence of redundant channel information on the network, effectively solve the problem of inaccurate features caused by channel redundancy in EEG spatial feature extraction, and improve the quality of spatial feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0076] Figure 1 is the overall network model structure diagram of the present invention;

[0077] Figure 2 is the data flow diagram of the overall network model of the present invention;

[0078] Figure 3 is the structure diagram of the TS-BiGRU model in the present invention;

[0079] Figure 4 is the structure of the dynamic channel attention mechanism model in the present invention;

[0080] Figure 5The coarse-grained classification results of the embodiment of the present invention are compared with those of EEGNet, AlexNet, ResNet, and Transformer on a self-collected dataset;

[0081] Figure 6 The fine-grained classification results of the embodiment of the present invention are compared with those of EEGNet, AlexNet, ResNet and Transformer on a self-collected dataset. DETAILED DESCRIPTION

[0082] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0083] Example 1: The present invention provides an EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism, the process is as follows Figure 1 , Figure 2 As shown, from Figure 1 It can be seen Figure 1 The raw EEG data first enters the bandfilter (bandpass filter) to filter the data, remove noise and retain signals in specific frequency bands. The data is then divided into two paths: one path enters the CAKE module for feature extraction to obtain spatial features, and the other path enters the TS-BiGRU module for feature extraction to obtain temporal features. The extracted spatial features and temporal features are spliced to achieve feature fusion, and then enter the softmax layer for prediction after passing through the feedforward neural network.

[0084] An EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism is carried out in the following steps:

[0085] S1 obtains initial EEG signal data, preprocesses the initial EEG signal data, and divides the preprocessed initial EEG signal data into a training set and a test set; the specific steps of S1 are as follows: S101: obtain initial EEG signal data, filter the initial EEG signal data using a frequency band filter, and obtain filtered initial EEG signal data; S102: process the filtered initial EEG signal data using a standardized algorithm, and divide the processed data into a training set and a test set.

[0086] In this embodiment, a band-pass filter with a frequency range of 0.5 Hz to 95 Hz is selected for the EEG signals. The reason is that this frequency range covers the main frequency components related to neural activities and emotions in EEG signals, such as delta waves (0.5 - 4 Hz), theta waves (4 - 8 Hz), alpha waves (8 - 13 Hz), beta waves (13 - 30 Hz), and gamma waves (30 - 95 Hz). By choosing this frequency range, the effective signals related to the analysis task can be maximally retained, while preventing meaningless or harmful high-frequency noise and low-frequency artifacts from being introduced into subsequent processing. After band-pass filtering, the signals are also normalized. Normalization can effectively eliminate the problem of inconsistent feature magnitudes caused by signal amplitude differences, making the data distribution more uniform and avoiding biases caused by signals in specific frequency bands having a large weight during neural network training. Through normalization, the original data is adjusted to a standard distribution with a mean of zero and a variance of one. This processing method not only improves the training efficiency of the network but also enhances the stability of the optimization process.

[0087] S2. Construct a TS-BiGRU module, input the EEG signal data of the training set into the TS-BiGRU module, perform time feature extraction, and obtain the time features of the EEG signal data of the training set.

[0088] The TS-BiGRU constructed in S2 includes: a forward GRU network containing the time-selective update gate z t a backward GRU network containing the time-selective update gate z t a forward GRU network containing the time-selective reset gate r t a backward GRU network containing the time-selective reset gate r t The functional expressions are respectively shown in formulas (1) - (4) ( Figure 3 ):

[0089] Among them, the forward GRU is mainly used to capture past information. Therefore, in the calculation of the update gate, it is desired to enhance the weight of the historical hidden state h t-1 while reducing the influence of the current input x t The forward GRU network containing the time-selective update gate z t is specifically adjusted as follows:

[0090]

[0091] where: z is the output of the update gate, f is forward, t is the time step t, is the output value of the forward GRU update gate at the current time step, σ is the sigmoid function, is the weight matrix applied to the current time step input in the forward GRU update gate, h t-1is the hidden state at the previous moment, is the weight matrix applied to the hidden state at the previous moment in the forward GRU update gate, x t is the input at the current time step, is the bias of the forward GRU update gate, λ is the decay coefficient, 0 < λ < 1;

[0092] Among them: The backward GRU mainly models future information, so the update gate is adjusted to make the weight of the current input x t larger, while the weight of the future hidden state h t+1 is relatively reduced. The backward GRU network containing the time-selective update gate z t is specifically adjusted as follows:

[0093]

[0094] Among them: z is the output of the update gate, b is backward, t is the time step t, is the output value of the backward GRU update gate at the current time step, σ is the sigmoid function, is the weight matrix applied to the input at the current time step in the backward GRU update gate, h t+1 is the hidden state at the next moment, is the weight matrix applied to the hidden state at the next moment in the backward GRU update gate, x t is the input at the current time step, is the bias of the backward GRU update gate, γ is the amplification coefficient, γ > 1;

[0095] The reset gate mainly controls the degree of forgetting of the current state h t of historical information and affects the short-term memory ability of the model. In TS-BiGRU, according to the different information requirements of the forward and backward directions, the reset gate is adjusted as follows.

[0096] Since the forward GRU mainly captures past information, it is hoped that the reset gate is easier to open, so that the historical hidden state h t-1 participates in the calculation to a greater extent. The forward GRU network containing the time-selective reset gate r t is specifically adjusted as follows:

[0097]

[0098] Among them: r is the output of the reset gate, f is forward, t is the time step t, is the output value of the forward GRU reset gate at the current time step, σ is the sigmoid function, is the weight matrix applied to the input at the current time step in the forward GRU reset gate, h t-1 is the hidden state at the previous moment, is the weight matrix applied to the hidden state at the previous moment in the forward GRU reset gate, and x t is the input at the current time step, is the bias of the forward GRU reset gate. In addition, α is the amplification factor, and α > 1;

[0099] The backward GRU mainly focuses on future information and hopes to reduce the dependence on h t+1 and enhance the ability to model local short-term information. The backward GRU network with the time-selective reset gate r t is specifically adjusted as follows:

[0100]

[0101] where: r is the output of the reset gate, b is backward, t is the time step t, is the output value of the backward GRU reset gate at the current time step, σ is the sigmoid function, is the weight matrix applied to the input at the current time step in the backward GRU reset gate, h t+1 is the hidden state at the next moment, is the weight matrix applied to the hidden state at the next moment in the backward GRU reset gate, x t is the input at the current time step, is the bias of the backward GRU reset gate, and β is the attenuation factor, 0 < β < 1.

[0102] The function expression for calculating the candidate hidden state in the TS-BiGRU module constructed in S2 is shown in Equation (5), and the function expression for updating the hidden state h t is shown in Equation (6), the function expression for calculating the backward candidate hidden state is shown in Equation (7), and the function expression for updating the backward hidden state is shown in Equation (8).

[0103]

[0104] where: is the candidate hidden state, tanh is the tanh function, W h , U h are the weight matrices, x t is the input at the current moment, r t is the reset gate, h t-1 is the hidden state at the previous moment, b h is the bias term, h t is the updated hidden state, z t is the update gate, ⊙ is the Hadamard product, is the backward input at the current moment, is a reverse reset gate, is the reverse hidden state at the next moment, is a reverse update gate, is the reverse candidate hidden state.

[0105] The functional expression output by the TS-BiGRU module constructed in S2 is as follows:

[0106] output,h n =TSBiGRU(e;θ) (9)

[0107] Where: e is the input sequence, c is the number of channels, d is the feature dimension, is the set of real numbers, θ is the trainable parameter in the TS-BiGRU model, output is the output sequence of the TS-BiGRU model at each time step, h n is the hidden state of the TS-BiGRU model at the last time step, and BiGRU is a bidirectional gated recurrent unit.

[0108] S3. Define the EFCM masking module, which is used to rank the channel importance of the EEG signal data in the training set;

[0109] The specific steps to rank the channel importance of the EEG signal data in the training set using the EFCM masking module in S3 are as follows:

[0110] S301: Let the EEG signal data in the training set be X, and denote the training set EEG signal data as a three-dimensional tensor Where, B is the batch size, C is the number of channels, t is the time step, and let the number of key channels to be selected by the EFCM masking module be k;

[0111] S302: Calculate the attention weights of each EEG signal data sample in the training set respectively. The attention weight of the p-th sample is w p,i , and the attention weight w p,i of the p-th sample. The functional expression is as follows:

[0112]

[0113] Where: X p,i is the time series of the i-th channel of the p-th sample, i is the channel index, p is the sample, and w p,i is the weight corresponding to the i-th channel in the p-th sample, is the transpose of the time series of the i-th channel of the p-th sample, and t is the time step;

[0114] S303: Use the softmax function to normalize the attention weights of each EEG signal data sample in the training set. For the attention weight \(w\) of the \(p\)-th sample, p,i , its normalized attention weight The function expression is as follows:

[0115]

[0116] where: is the weight value corresponding to the \(i\)-th channel of the \(p\)-th sample after normalization, \(w\) p,j is the original weight value corresponding to the \(j\)-th channel of the \(p\)-th sample, \(j\) is the index channel number, and \(\exp(.)\) is the natural exponential function;

[0117] S304: Normalize the attention weights of each EEG signal data sample in the training set. Sort the normalized attention weights \(w\) by channel, and determine the retained channel indices according to the key channel quantity value \(k\) in S301 to obtain the corresponding index set;

[0118] The normalized attention weight of the \(p\)-th sample in the training set corresponds to the sorted index set \(S\) P The function expression is as follows:

[0119] S p =\(\{S\) P,1 ,…,S P,k \}(12).

[0120] In this example, the core idea of the EFCM module is to rank the channel importance of each EEG signal, and only focus on the key channels that contribute the most to the task. Specifically, the EFCM module first calculates the attention weights of each channel through the attention mechanism to measure the importance of the channel in the current task. Subsequently, these weights are sorted from high to low, and a part of the channels with the top-ranked weights are selected and retained. These channels represent the information carriers most closely related to the target task. For those channels with lower weight rankings, the EFCM module will mask them and shield their information in subsequent processing to avoid interference from redundant information to the model performance. Through this design, the EFCM module can efficiently focus on the signal channels that are most critical to the task, thereby significantly improving the expression ability and robustness of the spatial feature extractor. Compared with the traditional full-channel processing method, this module can greatly reduce computational redundancy and perform more prominently in the expression of key features of the signal; The data of the EFCM masking module starts from the input attention weights, goes through calculating attention weights, normalization, selecting key channels, generating a mask matrix, and finally applying the mask to obtain the processed attention weights, realizing the channel screening function of the input signal.

[0121] S4. Construct the CAKE module with the EFCM masking module ( Figure 4 ), and input the training set EEG signal data into the CAKE module to extract spatial features and obtain the spatial features of the training set EEG signal data;

[0122] The CAKE module is used to dynamically extract the spatial features of the training set EEG signal data;

[0123] The CAKE module includes a multi-head attention mechanism, an EFCM module, and a soft routing mechanism;

[0124] The specific steps of S4 are as follows:

[0125] S401: Input the training set EEG signal data into the CAKE module, perform matrix multiplication, scaling, and softmax operations on the training set EEG signal data in sequence to obtain the initial attention weights, input the initial attention weights into the EFCM masking module in the CAKE module, calculate the attention weights corresponding to each path according to the channel differences masked by different paths, and multiply the attention weights corresponding to multiple paths by the path factor r k obtained by the soft routing mechanism to get the path-enhanced attention weights;

[0126] S402: Multiply the path-enhanced attention weights by the value matrix V to obtain the output result of a single attention head, splice the output results of all H attention heads to form a multi-head attention output matrix, and perform linear projection processing on the multi-head attention output matrix to obtain the spatial features of the training set EEG signal data.

[0127] The specific steps of S401 are as follows:

[0128] Step S4011: Input each EEG signal data sample in the training set into the CAKE module, use the CAKE module to perform a linear transformation on the channel features of each data sample to obtain the query matrix Q, the key matrix K, and the value matrix V. For the p-th sample, the query matrix Q p is expressed by the function formula as shown in Equation (13), and the key matrix K p is expressed by the function formula as shown in Equation (14), and the value matrix V p is expressed by the function formula as shown in Equation (15):

[0129] Q p = X p W Q (13)

[0130] K p = X p W K (14)

[0131] V p =X p W V (15)

[0132] Where: Q p is the query matrix corresponding to the p-th sample, K p is the key matrix corresponding to the p-th sample, V p is the value matrix corresponding to the p-th sample, X p is the input data sample, W Q 、W K 、W V are trainable parameter matrices, d model =H×d head , H is the number of attention heads, d head is the dimension of each attention head, X p is the p-th sample data, C is the number of channels, d model is the feature dimension, is the set of real numbers;

[0133] Step S4012: Split the query matrix Q, key matrix K, and value matrix V corresponding to each EEG signal data sample in the training set into H attention heads. For the query matrix Q p , key matrix K p , and value matrix V p The split query matrix Q p is shown in Equation (16), the split key matrix K p is shown in Equation (17), and the split value matrix V p is shown in Equation (18):

[0134] Q p,h =Q p [:,(h - 1)d head :hd head (16)

[0135] K p,h =K p [:,(h - 1)d head :hd head (17)

[0136] V p,h =V p [:,(h - 1)d head :hd head (18)

[0137] Where: Q p,h is the query matrix of the h-th attention head of the p-th sample, Kp,h is the key matrix of the h-th attention head for the p-th sample, V p,h is the value matrix of the h-th attention head for the p-th sample, where h is the number of the attention head;

[0138] Step S4013: Input the attention score A corresponding to each EEG signal data sample in the training set h into the EFCM masking module to obtain the attention score matrix of the corresponding path. For the A corresponding to the p-th sample p,h , its functional expression is as shown in (19). Input the p-th sample into the EFCM module to obtain the attention score matrix of the path obtained by the EFCM masking module for the p-th sample at the h-th attention head as shown in Equation (20)

[0139]

[0140] where: A p,h is the attention score corresponding to the p-th sample, softmax is the softmax function, Q p,h is the query matrix of the p-th sample at the h-th attention head, is the transpose of the key matrix of the p-th sample at the h-th attention head, and T is the transpose;

[0141] MA p,h = EFCM(A p,h ) (20)

[0142] where: MA p,h is the attention score matrix of multiple paths obtained by the EFCM masking module for the p-th sample at the h-th attention head;

[0143] Multiply the attention score matrix corresponding to multiple paths by the path factor r k to obtain the path-enhanced attention score matrix. Multiply the path-enhanced attention score matrix corresponding to each EEG signal data sample in the training set by the value matrix to obtain the weighted value matrix of each data sample. For the weighted value matrix of the p-th sample at the h-th attention head, it is as shown in Equation (21):

[0144] Z p,h = MA p,h r p,h,k V p,h (21)

[0145] where: Z p,h is the weighted value matrix of the p-th sample at the h-th attention head, r p,h,k is the path factor of the p-th sample at the h-th attention head, V p,h is the value matrix of the p-th sample at the h-th attention head, and k is the number of paths.

[0146] S402: Multiply the path-enhanced attention weights with the value matrix V to obtain the output result of a single attention head, concatenate the output results of all H attention heads to form a multi-head attention output matrix, and perform linear projection processing on the multi-head attention output matrix to obtain the spatial features of the training set EEG signal data.

[0147] In this embodiment, in the calculation process of the CAKE module, it is first necessary to multiply the channel mask generated by the EFCM mechanism with the routing factor r calculated by the soft routing mechanism. The EFCM mechanism is responsible for generating different masks for multiple paths according to the distribution of attention weights, and each path corresponds to a set of channel selection schemes. The soft routing mechanism, through a data-driven method, assigns a dynamic weight factor r to each path, and these weight factors reflect the network's judgment on the importance of different paths. By multiplying the mask with the routing factor r, a set of dynamically adjusted attention weights can be finally obtained. Next, the dynamically adjusted attention weights will perform matrix multiplication with the Value. This process is a key step in the attention mechanism, and its main purpose is to perform a weighting operation on the Value. Specifically, the attention weights determine the importance of each channel in the input features. Therefore, after multiplying with the Value, important channels will be amplified, while secondary or irrelevant channels will be suppressed. In this way, the network can more accurately extract the information meaningful for the current task in the input data. After the weighting operation, the output result of the current attention head will be finally generated.

[0148] S5. Concatenate the time features obtained in S2 and the spatial features obtained in S4 to obtain fused features;

[0149] S6. Transmit the fused features to a feed-forward neural network to obtain the probability distributions of various categories, and then use the softmax function to normalize the probability distributions to obtain the classification results of the training set EEG signal data.

[0150] In this embodiment, subsets of the publicly available ImageNet-EEG dataset and the self-collected remote sensing image visual stimulation EEG dataset are respectively input into the neural network model to obtain two sets of EEG visual classification results. The publicly available subset of the ImageNet-EEG dataset and the self-collected remote sensing image-EEG dataset are respectively used to verify the classification performance of the model. The construction method of the self-collected remote sensing image visual stimulation EEG dataset in this embodiment is as follows:

[0151] Step a: Obtain a visual stimulation dataset and perform standardization processing on the picture data to ensure the consistency of visual characteristics.

[0152] Step a.1: Screen the dataset

[0153] A remote sensing image dataset suitable for this example was collected and screened. By systematically searching military image databases, remote sensing imaging platforms (such as UC Merced LandUse, NWPU-RESISC45, and DOTA), and open-source aerial imagery resources, we obtained remote sensing images from eight categories. For each category, we rigorously screened 100 high-quality images, constructing a standard image dataset totaling 800 images.

[0154] Because the viewing angles and lighting conditions of the images in some data sets vary greatly, the quality of the EEG signals induced by visual stimulation may be affected. To ensure the scientific nature of the experiment, this embodiment conducts a strict quality assessment of the candidate images: an image clarity detection algorithm is used to evaluate the image quality, excluding blurry or low-resolution samples; a histogram analysis is performed to ensure that the lighting conditions are within an acceptable range; and standardized requirements are imposed on the proportion of the image subject in the image (controlled between 40% and 60% of the image area). Finally, samples that meet the requirements are selected as experimental stimulus materials.

[0155] Step a.2: Stimulus matching

[0156] To eliminate interference from irrelevant variables, this embodiment uniformly resizes the image processed in step 1 to 500×500 pixels, then uses the LAB color space for brightness normalization. The visual feature extraction network verifies the distinguishability of each category in the visual feature space, ensuring that the interference category maintains a moderate visual similarity with the target category.

[0157] In this embodiment, four categories are set for both coarse-grained and fine-grained experiments, and all category images undergo the above-mentioned standardized processing flow to ensure the scientificity and comparability of the experimental stimulus materials.

[0158] Step a.3: Experimental timing matching

[0159] To ensure that the timing parameters of experiments of different granularities are strictly consistent and to avoid cognitive bias due to differences in time processes, this embodiment adopts a standardized experimental timing design, and all stimulus presentation and interval times are achieved through precise time control.

[0160] In this embodiment, the four-category EEG acquisition experiment for visual remote sensing images adopts a dual-granularity design, comprising two experimental layers (sessions), corresponding to coarse-grained and fine-grained classification, respectively. Each experimental layer contains four experimental blocks, each corresponding to 100 images of a category.

[0161] The experimental timing parameters are as follows: 1. Trial process: After the subject presses any key, the stimulus presentation starts after a 5000 ms delay; the stimulus picture is presented for 1000 ms; after the stimulus ends, a 500 ms black screen + white "+" fixation point is displayed; 2. Experimental interval control: A mandatory break of 60000 ms (1 minute) is taken between two experimental layers; the subject needs to press a key to confirm before the start of each experimental block; 3. Total duration: 1,266,000 ms (21.1 minutes), and the time error is controlled within ±1 ms (accurate timing using software such as PsychoPy). This design ensures the consistency of the timing of experiments at different granularities, avoids cognitive biases, and at the same time optimizes the signal-to-noise ratio of EEG signals, providing reliable data for subsequent EEG analysis.

[0162] Step b: EEG data acquisition.

[0163] Step b.1 Paradigm design

[0164] The experimental paradigm based on visual cognitive neuroscience can provide a reliable EEG acquisition framework. By systematically retrieving the literature in the fields of visual ERP research and aviation image cognition, including neuroscience papers and psychological reports, and deeply analyzing the design principles of existing visual classification paradigms. On this basis, with key reference to the experimental paradigm most relevant to the research objectives of the embodiments of the present invention (EEG classification of remote sensing aircraft images), adaptive modifications are made to its core parameters.

[0165] In this example, the paradigm design uses a dual-block structure as the main framework, corresponding to coarse-grained and fine-grained four-classification tasks respectively, where the four categories of the fine-grained focus on the four fine distinctions of one of the four categories of the coarse-grained. By strictly controlling the stimulus duration (1000 ms presentation time) and trial interval (500 ms black screen + fixation point), it is ensured that the subject can not only complete sufficient visual feature extraction but also maintain a stable attention level. The fixation point is designed with a high-contrast white "+", and its presentation time and spatial characteristics have been verified by parametric tests to effectively stabilize visual fixation. After pre-experiment verification, this dual-block paradigm can stably induce N170 and P300 components related to visual classification, and the total experimental duration is accurately controlled within the design goal of 21.1 minutes, and it is finally determined as the formal experimental plan.

[0166] Step b.2: Collect EEG data of visual stimuli

[0167] On the basis of standardizing visual stimuli in the previous steps, a dual-block paradigm is used for the formal experiment to verify the subject's category recognition ability of remote sensing images and provide reliable data for subsequent network training. The experimental design strictly follows the research norms of cognitive neuroscience, and the data quality is ensured by precisely controlling the experimental parameters.

[0168] Experimental design description:

[0169] In this embodiment, a hierarchical block design was adopted in the experiment. This design presented visual stimuli at different granularity levels in blocks, with each block presenting aircraft images of the same category or subcategory in a concentrated manner, so as to facilitate the comparison of the activity differences in the visual cortex of the brain at different classification levels. In the experiment, the stimulus presentation time was strictly controlled at 1000 ms, and this duration was verified by preliminary experiments to meet the requirements of visual feature extraction. After each stimulus presentation, a central fixation point "+" was set for 500 ms as the inter-trial interval, which not only ensured that the subjects obtained the necessary visual rest but also maintained a stable attention level. No behavioral feedback link was set up throughout the experiment, and the subjects were guided to perform visual classification tasks through standardized instructions, minimizing the interference of external factors on the cognitive process to the greatest extent. The experiment included two classification levels and a total of 8 visual categories. Each block contained 100 trials, for a total of 800 trials, ensuring the acquisition of electroencephalogram data with statistical significance. The physical parameters such as the brightness and contrast of all visual stimuli were standardized to ensure the fairness and consistency of the experimental conditions.

[0170] Equipment and experimental environment:

[0171] A 64-channel Biosemi ActiveTwo electroencephalogram acquisition system was used as the experimental equipment, and the sampling rate was set at 1024 Hz. The electrode layout strictly followed the international 10-20 system standard. The laboratory environment was kept at a constant temperature and humidity, and the light intensity was controlled between 150 - 200 lux to ensure the consistency of visual stimulus presentation.

[0172] The experimental steps are as follows:

[0173] Before the formal experiment started, the technical staff correctly wore the electroencephalogram cap for the subject using professional conductive paste. The tightness of the electroencephalogram cap needed to be adjusted appropriately to ensure both signal quality and the comfort of the subject. The subject needed to sit upright, with the head naturally relaxed on the headrest, and look straight ahead at the center position of the display, maintaining a fixed viewing distance of 70 cm. During the experiment, the subject was required to maintain a natural blinking frequency. When the stimulus pictures were presented, they should concentrate on observing the aircraft features, but avoid deliberately staring or blinking frequently. During each block interval, the subject could relax the eyes moderately. The subject was specifically reminded to keep the body still throughout the experiment and avoid behaviors that might generate electromyogram interference, such as head movement, limb movement, and swallowing. All cognitive judgment processes were completed inside the brain without any external behavioral feedback. The subject needed to ensure sufficient sleep 24 hours before the experiment and avoid drinking alcohol or beverages containing caffeine to ensure that the brain was in the best cognitive state. Through such strict operating specifications, high-quality visually evoked electroencephalogram signals were ensured to be collected.

[0174] In this embodiment, the self - collected EEG dataset of visual stimuli from remote sensing images is divided into three parts: a training set, a validation set, and a test set, with the division ratios being 80%, 10%, and 10% respectively. This division scheme is based on the following considerations: 1) Retaining a sufficient sample size for model development and optimization; 2) Ensuring that the test set has statistical significance; 3) Achieving reliable hyperparameter tuning through the validation set during the training process. The training set further adopts a stratified five - fold cross - validation method, that is, on the premise of keeping the sample ratio of each category unchanged, the training set is evenly divided into 5 subsets. In turn, one of the subsets is used as the validation set, and the remaining 4 are used as the training set, cycling 5 times to ensure that each piece of data can participate in the validation. This division method not only avoids the randomness impact of data division but also makes full use of limited data resources. Its functional expression is shown in Equation (22):

[0175]

[0176] where: K = 5 is five - fold cross - validation, and score i is the model performance metric (such as accuracy) obtained in the i - th validation.

[0177] This embodiment adopts a systematic hyperparameter optimization strategy to determine the best parameter configuration through experimental verification to optimize the model training process, including the selection of hyperparameters such as batch size, learning rate, and optimizer. The specific parameter settings are as follows:

[0178] (1) Batch Size: The batch size is determined to be 32 through grid search. While ensuring a video memory utilization rate of 85% ± 3% for the NVIDIA RTX 3090 graphics card, the gradient variance is stably controlled within the range of 0.02 - 0.05 to avoid out - of - memory errors.

[0179] (2) Learning Rate and Scheduler: The learning rate adopts a dynamic adjustment scheme. The initial value is set to the commonly used 0.001, and the CosineAnnealing scheduler is applied to achieve periodic adjustment. The function of the learning rate scheduler is to adjust the learning rate according to the training progress. After every set number of n epochs, the learning rate is decreased. Through this strategy, rapid convergence can be achieved in the initial stage of training, and fine - tuning can be carried out in the later stage to improve the model accuracy. The functional expression of the CosineAnnealingWarmRestarts scheduler is shown in Equation (22):

[0180]

[0181] where: η t is the learning rate at the current time step t, η min is the minimum value (lower limit) of the learning rate, η max is the initial value (upper limit) of the learning rate, and T curis the current step (epoch / iteration), T max is the total cycle length (preset decay steps).

[0182] (3) Loss function: In this embodiment, the cross-entropy loss function is used as the optimization objective for model training. This function can effectively measure the difference between the model's predicted probability distribution and the true label. Its mathematical expression is:

[0183]

[0184] where: C is the number of classes, y i is the true label of the sample, and p i is the probability of the class predicted by the model.

[0185] (4) Optimizer: In this embodiment, the Adam optimizer (Adaptive Moment Estimation) is selected as the optimizer, which combines the adaptive learning rate characteristics of the first-order momentum and the second-order momentum.

[0186] (5) Regularization: To prevent the model from overfitting, this embodiment adopts a regularization strategy that combines L2 weight decay (coefficient 0.01) and Dropout (ratio 0.1). Among them, Dropout randomly masks some neurons during the training process and uses overall scaling during the testing phase. All hyperparameter settings are determined through grid search and their effectiveness is verified on the validation set.

[0187] In summary, the method proposed in this application adopts a parallel structure in the design of the time feature extractor and the spatial feature extractor, rather than the traditional serial structure. The parallel structure can process both time and space features simultaneously, significantly improving the running efficiency, and at the same time avoiding the time overhead problem caused by the gradual extraction of features in the serial processing process. This design makes the processing process faster and provides a more efficient solution for tasks that require real-time response. In addition, the parallel structure shows significant advantages in the independent extraction and expression of features, effectively avoiding the mutual interference between different patterns of features, significantly reducing the possibility of feature confusion, and thus enhancing the purity and expression ability of the extracted features. During the feature extraction process, the time feature extraction and the spatial feature extraction can run independently and maintain relative independence from each other. This independence makes the method more robust and flexible when processing data. It not only improves the overall computational efficiency but also significantly enhances the adaptability and generalization ability of the method to features. Whether it is the processing of time-dominated dynamic signals or the analysis of space-dominated static features, this method can show good performance.

[0188] The masking mechanism in the method proposed in this application is different from the traditional fixed masking mechanism. This method introduces multiple paths in the network. The difference between the paths lies in that the masks corresponding to each path retain different numbers of channels. Therefore, in this method, instead of explicitly specifying the number of channels to be retained in each layer of the network artificially, multiple paths are provided for the network to autonomously select the optimal combination according to the characteristics of the input data during the training process. The role of the soft routing mechanism is to allocate appropriate path weights to the input data. By learning the matching relationship between the input data and the paths, multiple paths are weighted and combined. This dynamic selection mechanism enables the network to simultaneously retain important high-weight channels and reasonably mask unimportant or potentially interfering low-weight channels. Taking the first layer as an example, although four paths with fixed numbers of channels are provided, the final path selection is not fixed but dynamically completed through the soft routing mechanism. The network can flexibly combine these paths according to the needs in different samples, greatly improving the generalization performance of the model and its adaptability to complex scenarios, and can also effectively capture the diverse features in the data.

[0189] To systematically verify the effectiveness of the method proposed in this application, the method proposed in this application was comprehensively tested on public datasets and self-collected datasets.

[0190] Regarding public datasets, three representative subsets of ImageNet were selected for testing. Each subset contains five categories of visual targets (50 professionally screened natural images for each category). These images have significant differences in visual features such as color, shape, texture, and background complexity, and can comprehensively evaluate the generalization ability of the model in different natural visual scenarios. Regarding the self-collected dataset, a professionally constructed remote sensing image dataset (8 categories, 800 images) was used for testing. All images have undergone strict brightness and contrast normalization processing, involving challenging factors in real scenarios such as multi-scale target recognition, variable background interference, complex lighting conditions, and partial occlusion, and can comprehensively evaluate the generalization ability of the model in remote sensing images of complex scenarios. The experimental results are shown in Table 1-2 and Figure 5 、 6 as follows.

[0191] Table 1 Performance test results of the method proposed in the present invention in the ImageNet EEG visual classification task

[0192] ImageNet subset Accuracy F1 score Five-class subset 1 0.84 0.85 Five-class subset 2 0.88 0.89 Five-class subset 3 0.92 0.92

[0193] It is found that the accuracy of the method proposed in this application for EEG visual classification on subset 1 is 0.84, on subset 2 is 0.88, and on subset 3 is 0.92. Compared with traditional classification methods, the significant improvement of the method proposed in this invention in terms of accuracy and F1 score reflects its excellent ability in extracting key features of EEG signals and distinguishing different natural image visual targets.

[0194] Table 2 Performance test results of the method proposed in this invention in the EEG visual classification task of remote sensing images (self - collected dataset)

[0195] Task granularity Accuracy F1 score Coarse-grained four-classification 0.84 0.84 Fine-grained four-classification 0.81 0.82

[0196] It is found that the accuracy of the method proposed in this application for EEG visual classification in coarse - grained four - classification is 0.84, and in fine - grained four - classification is 0.81. Compared with traditional classification methods, the significant improvement of the method proposed in this invention in terms of accuracy and F1 score reflects its excellent ability in extracting key features of EEG signals and distinguishing different remote sensing image visual targets.

[0197] Figure 5 It shows the comparison of the coarse - grained classification results of the method proposed in this invention with EEGNet, AlexNet, ResNet and Transformer models on the self - collected dataset. From Figure 5 it can be seen that the method proposed in this invention has the highest accuracy, reaching 0.84, and performs extremely well in related tasks, with better performance than the other comparison models. The EEGNet model has the lowest accuracy, only 0.76, and performs poorly in this task compared with other models.

[0198] Figure 6 It shows the comparison of the fine - grained classification results of the method proposed in this invention with EEGNet, AlexNet, ResNet and Transformer models on the self - collected dataset. From Figure 6 it can be seen that the accuracy of the method proposed in this invention is 0.81, and the accuracy of the EEGNet model is only 0.74, with the worst effect and a large gap from other models.

[0199] Figure 5 And Figure 6 Ours in [reference] represents the EEG visual classification method based on TS - BiGRU and dynamic channel attention mechanism.

[0200] In summary, the method proposed in the present invention maintains the optimal performance on datasets of two granularities. Even compared with the sub-optimal model (Transformer), it has a 1 percentage point improvement, fully demonstrating the superiority and stability of the method proposed in the present invention.

[0201] In addition, the experimental results in Table 1 and Table 2 also prove that the method proposed in the present invention has good adaptability and robustness to data. On different datasets, whether it is the complexity of image features or the degree of background change, the method of the present invention shows stable classification performance. This adaptability and robustness stem from the parallel extraction of spatio-temporal features designed by the method of the present invention and the dynamic channel attention mechanism's profound understanding of the characteristics of EEG signals and full consideration of the diverse requirements of classification tasks. Facing different data distributions and visual features, the method of the present invention can quickly adapt to the data and maintain a high classification performance.

[0202] The present invention also includes an electronic device, including a memory and a processor. The memory is used to store various computer program instructions, and the processor is used to execute the computer program instructions to complete all or part of the above steps; the electronic device can communicate with one or more external devices, and can also communicate with one or more devices that enable users to interact with the electronic device, and / or communicate with any device that enables the electronic device to communicate with one or more other computing devices. The electronic device can also communicate with one or more networks (such as local area networks, wide area networks, and / or public networks) through a network adapter.

[0203] The present invention also includes a computer-readable medium storing a computer program, which can be executed by a processor. The computer-readable medium can include, but is not limited to, magnetic storage devices, optical discs, digital versatile discs, smart cards, and flash memory devices. In addition, the readable storage medium of the present invention can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" includes, but is not limited to, wireless channels and various other media (and / or storage media) that can store, contain, and / or carry code and / or instructions and / or data.

[0204] Each embodiment in this specification is described in a related manner. For the same or similar parts between the embodiments, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments.

[0205] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism, characterized in that, The specific steps are as follows: S1. Obtain the initial EEG signal data, preprocess the initial EEG signal data, and divide the preprocessed initial EEG signal data into a training set and a test set; S2. Construct a TS-BiGRU module, input the EEG signal data of the training set into the TS-BiGRU module, perform time feature extraction, and obtain the time features of the EEG signal data of the training set; S3. Define an EFCM masking module, which is used to rank the channel importance of the EEG signal data of the training set; S4. Construct a CAKE module containing the EFCM masking module, input the EEG signal data of the training set into the CAKE module, perform spatial feature extraction, and obtain the spatial features of the EEG signal data of the training set; The CAKE module is used to dynamically extract the spatial features of the EEG signal data of the training set; The CAKE module includes a multi-head attention mechanism, an EFCM module, and a soft routing mechanism; S5. Concatenate the time features obtained in S2 and the spatial features obtained in S4 to obtain fused features; S6. Transmit the fused features to a feed-forward neural network to obtain the probability distribution of each category, and then use the softmax function to normalize the probability distribution to obtain the classification result of the EEG signal data of the training set.

2. The EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism according to claim 1, wherein The specific steps of S1 are as follows: S101: Obtain the initial EEG signal data, filter the initial EEG signal data using a band-pass filter to obtain the filtered initial EEG signal data; S102: Process the filtered initial EEG signal data using a standardization algorithm, and divide the processed data into a training set and a test set.

3. The EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism according to claim 1, wherein The TS-BiGRU module constructed by S2 includes: a forward GRU network with a time-selective update gate z t a backward GRU network with a time-selective update gate z t a forward GRU network with a time-selective reset gate r t a backward GRU network with a time-selective reset gate r t The functional expressions are shown in formulas (1) to (4) respectively: Where: z is the output of the update gate, f is the forward direction, t is the time step, is the output value of the forward GRU update gate at the current time step, σ is the sigmoid function, is the weight matrix applied to the input at the current time step in the forward GRU update gate, h t-1 is the hidden state at the previous moment, is the weight matrix applied to the hidden state at the previous moment in the forward GRU update gate, x t is the input at the current time step, is the bias of the forward GRU update gate, λ is the decay coefficient, 0 < λ < 1; where: b is backward, is the output value of the backward GRU update gate at the current time step, is the weight matrix applied to the input at the current time step in the backward GRU update gate, h t+1 is the hidden state at the previous moment, is the weight matrix applied to the hidden state at the previous moment in the backward GRU update gate, is the bias of the backward GRU update gate, γ is the amplification factor, γ > 1; where: r is the reset gate output, which is the output value of the forward GRU reset gate at the previous time step, is the weight matrix applied to the input at the current time step in the forward GRU reset gate, is the weight matrix applied to the hidden state at the previous time step in the forward GRU reset gate, is the bias of the forward GRU reset gate, and α is the amplification factor, where α > 1; Wherein: is the output value of the backward GRU reset gate after the current time step, is the weight matrix applied to the input of the current time step in the backward GRU reset gate, is the weight matrix applied to the hidden state of the previous time step in the backward GRU reset gate, is the bias of the backward GRU reset gate, and β is the attenuation coefficient, where 0 < β < 1.

4. The EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism according to claim 1, characterized in that The function expression for calculating the candidate hidden state in the TS-BiGRU module constructed in S2 is shown in Equation (5), and the function expression for updating the hidden state h t is shown in Equation (6), and the function expression for calculating the backward candidate hidden state is shown in Equation (7), and the function expression for updating the backward hidden state is shown in Equation (8): Wherein: is the candidate hidden state, tanh is the tanh function, W h , U h is the weight matrix, x t is the input at the current moment, r t is the reset gate, h t-1 is the hidden state at the previous moment, b h is the bias term, h t is the updated hidden state, z t is the update gate, ⊙ is the Hadamard product, is the reverse input at the current moment, is the reverse reset gate, is the reverse hidden state at the next moment, is the reverse update gate, is the reverse candidate hidden state.

5. The EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism according to claim 1, characterized in that, The output function expression of the TS-BiGRU module constructed in S2 is as follows: output, h n = TSB i GRU(e; θ) (9) where: e represents the input sequence, c is the number of channels, d is the feature dimension, is the set of real numbers, θ is the trainable parameter in the TS-BiGRU model, output is the output sequence of the TS-BiGRU model at each time step, h n is the hidden state of the TS-BiGRU model at the last time step, and BiGRU is the bidirectional gated recurrent unit.

6. The EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism according to claim 1, characterized in that The specific steps of using the EFCM masking module to rank the channel importance of the EEG signal data of the training set in S3 are as follows: S301: Let the EEG signal data of the training set be X, and denote the EEG signal data of the training set as a three-dimensional tensor where B is the batch size, C is the number of channels, t is the time step, and let the number of key channels to be selected by the EFCM masking module be k; S302: Calculate the attention weights of each EEG signal data sample in the training set respectively, where the attention weight of the p-th sample is w p,i , and the attention weight w p,i of the p-th sample has the following functional expression: Where: X p,i is the time series of the i-th channel of the p-th sample, where i is the channel index, p is the sample, and w p,i is the weight corresponding to the i-th channel in the p-th sample, is the transpose of the time series of the i-th channel of the p-th sample, where t is the time step; S303: Use the softmax function to normalize the attention weights of each EEG signal data sample in the training set. For the attention weight w of the p-th sample p,i , its normalized attention weight The functional expression is as follows: Wherein: is the weight value corresponding to the i-th channel of the p-th normalized sample, w p,j is the original weight value corresponding to the j-th channel of the p-th sample, j is the index channel number, and exp(.) is the natural exponential function; S304: Normalize the attention weights of each EEG signal data sample in the training set, sort the normalized attention weights w by channel, and determine the retained channel indices according to the key channel quantity value k described in S301 to obtain the corresponding index set; The attention weights after normalization of the p-th sample in the training set The corresponding sorted index set S P The function expression is as follows: S p = {S P,1 , …, S P,k} (12).

7. The EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism according to claim 1, characterized in that, The specific steps of S4 are as follows: S401: Input the EEG signal data of the training set into the CAKE module, perform matrix multiplication, scaling, and softmax operations on the EEG signal data of the training set in sequence to obtain the initial attention weights, input the initial attention weights into the EFCM masking module in the CAKE module, calculate the attention weights corresponding to each path according to the channel differences masked by different paths, and multiply the attention weights corresponding to multiple paths by the path factor r obtained by the soft routing mechanism k to obtain the path-enhanced attention weights; S402: Multiply the path-enhanced attention weights by the value matrix V to obtain the output result of a single attention head, concatenate the output results of all H attention heads to form a multi-head attention output matrix, and perform linear projection processing on the multi-head attention output matrix to obtain the spatial features of the EEG signal data of the training set.

8. The EEG visual classification method based on TS-BiGRU and dynamic channel attention mechanism according to claim 7, characterized in that, The specific steps of S401 are as follows: Step S4011: Input each EEG signal data sample in the training set into the CAKE module, and use the CAKE module to perform a linear transformation on the channel features of each data sample to obtain a query matrix Q, a key matrix K, and a value matrix V. For the p-th sample, the query matrix Q p has a functional expression as shown in Equation (13), and the key matrix K p has a functional expression as shown in Equation (14), and the value matrix V p has a functional expression as shown in Equation (15): Q p = X p W Q (13) K p = X p W K (14) V p = X p W V (15) Where: Q p is the query matrix corresponding to the p-th sample, K p is the key matrix corresponding to the p-th sample, V p is the value matrix corresponding to the p-th sample, X p is the input data sample, W Q 、W K 、W V are trainable parameter matrices, d model = H × d head , where H is the number of attention heads, d head is the dimension of each attention head, X p is the p-th sample data, C is the number of channels, d model is the feature dimension, is the set of real numbers; Step S4012: Split the query matrix Q, key matrix K, and value matrix V corresponding to each EEG signal data sample in the training set into H attention heads respectively. For the query matrix Q corresponding to the p-th sample p , the key matrix K p , and the value matrix V p The split query matrix Q p As shown in Equation (16), the split key matrix K p As shown in Equation (17), the split value matrix V p As shown in Equation (18): Q p,h = Q p [:,(h - 1)d head :hd head (16) K p,h = K p [:,(h - 1)d head :hd head (17) V p,h = V p [:,(h - 1)d head :hd head (18) Where: Q p,h is the query matrix of the h-th attention head for the p-th sample, K p,h is the key matrix of the h-th attention head for the p-th sample, V p,h is the value matrix of the h-th attention head for the p-th sample, and h is the number of the attention head; Step S4013: Transfer the attention score A corresponding to each EEG signal data sample in the training set h to the EFCM masking module to obtain the attention score matrix of the corresponding path. For the A corresponding to the p-th sample p,h , its functional expression is shown in (19). Transfer the p-th sample to the EFCM module to obtain the attention score matrix of the path obtained by the EFCM masking module for the p-th sample at the h-th attention head as shown in Equation (20). Where: A p,h is the attention score corresponding to the p-th sample, softmax is the softmax function, and Q p,h is the query matrix of the p-th sample for the h-th attention head, is the transpose of the key matrix of the p-th sample for the h-th attention head, and T is the transpose; MA p,h = EFCM(A p,h ) (20) Where: MA p,h is the attention score matrix of multiple paths obtained by the EFCM masking module for the p-th sample at the h-th attention head; Multiply the attention score matrices corresponding to multiple paths by the path factor r k to obtain an attention score matrix with enhanced paths. Multiply the attention score matrix with enhanced paths corresponding to each EEG signal data sample in the training set by the value matrix to obtain a weighted value matrix for each data sample. The weighted value matrix for the p-th sample corresponding to the h-th attention head is shown in Equation (21): Z p,h = MA p,h r p,h,k V p,h (21) Where: Z p,h is the weighted value matrix of the p-th sample in the h-th attention head, r p,h,k is the path factor of the p-th sample in the h-th attention head, V p,h is the value matrix of the p-th sample in the h-th attention head, and k is the number of paths.

9. An electronic device, characterized in that, It includes a processor, a memory, and a communication bus. Among them, the processor and the memory complete mutual communication through the communication bus; The memory is used to store computer programs; The processor, when executing the programs stored on the memory, implements the method steps described in any one of claims 1-8.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, and when the computer program is executed by the processor, it implements the method steps described in any one of claims 1-8.

Images