An eeg classification method of a hybrid attention prototype network fused with feature weights

Abstract

The application discloses a fusion feature weight mixed attention prototype network electroencephalogram classification method, comprising: collecting electroencephalogram data; preprocessing the collected EEG original data, extracting differential entropy features, standardizing and sampling the extracted features, and generating a support set and a query set; inputting the preprocessed features into a prototype network, extracting source domain / target domain embedding representations, and calculating feature weights and channel-frequency band weights through a double attention layer; calculating a prototype representation, fusing feature attention weights and channel-frequency band attention weights, and obtaining a final weighted distance; introducing a Grad-CAM visualization mechanism to analyze the distribution consistency of weight distribution; inputting the fused representation into a classifier, calculating a classification loss according to a prediction score, and outputting a cross-period EEG electroencephalogram classification result; and the application solves the problems of traditional 'black box' and weak domain adaptation, effectively improves the cross-period electroencephalogram classification accuracy, robustness and explainability.

Description

Technical Field

[0001] This invention relates to the fields of brain-computer interfaces, intelligent signal processing, and deep learning technologies, and specifically to a hybrid attention prototype network EEG classification method that integrates feature weights. Background Technology

[0002] EEG, as a non-invasive brainwave monitoring technology, has advantages such as millisecond-level temporal resolution, low cost, and ease of deployment, and has become a core data source for EEG classification. However, EEG signals suffer from significant cross-temporal domain shifts. The EEG signals of the same subject at different time periods (e.g., several days apart) can change in feature distribution due to physiological state (fatigue, attention), environmental interference (noise, electromagnetic interference), or equipment differences. At the same time, the channel-band specificity of EEG signals (different brain regions and frequency bands respond differently to tasks) and low signal-to-noise ratio further increase the difficulty of EEG classification.

[0003] Existing cross-time EEG classification methods have the following shortcomings:

[0004] 1. Insufficient domain adaptability: Traditional methods often rely on single domain adaptation loss (such as maximum mean difference MMD), without combining feature-level consistency constraints, making it difficult to effectively alleviate cross-time distribution shifts;

[0005] 2. Single attention mechanism: Most methods only focus on single attention in the feature dimension or channel dimension, ignoring the multi-dimensional collaborative optimization of "feature-channel-frequency band", and cannot fully explore the key EEG components related to EEG classification;

[0006] 3. Lack of interpretability: Deep learning models are often regarded as "black boxes," lacking visualization analysis of attention weight allocation and key brain regions / bands, making it difficult to verify the rationality of the model's decisions.

[0007] 4. Low sample utilization: In cross-time scenarios, the number of labeled samples is limited. Traditional few-shot learning methods have not designed a special sampling strategy for the 62-channel × 5-band structure of EEG, which can easily lead to sample class imbalance. Summary of the Invention

[0008] The purpose of this invention is to provide a hybrid attention prototype network EEG classification method that integrates feature weights. By multi-dimensional attention fusion, cross-time domain adaptive constraints, and visualization interpretation, it improves the accuracy and robustness of EEG classification in cross-time domain scenarios, thereby solving the problems mentioned in the background art.

[0009] To address the aforementioned technical problems, this invention provides the following technical solution: a hybrid attention prototype network EEG classification method that integrates feature weights, comprising the following operational steps:

[0010] Step S1: Collect EEG data from multiple subjects using specific tasks at different time periods.

[0011] Step S2: Preprocess the collected raw EEG data, extract differential entropy features, and generate support sets and query sets by constructing datasets for the source and target domains and standardizing and sampling the extracted features.

[0012] Preferably, the specific implementation steps include:

[0013] Step S21: Perform preprocessing operations such as filtering, denoising, and zero-mean normalization on the original EEG signal;

[0014] Step S22: Extract differential entropy features from the differential entropy feature file of the public dataset;

[0015] Step S23: Construct a source domain-target domain training / validation / test split, and use balanced sampling to keep the number of samples for each classification category consistent.

[0016] Step S3: Input the preprocessed features into the prototype network, extract the embedding representations of the source / target domain, and calculate the feature weights and channel-band weights through a dual attention layer.

[0017] Preferably, the specific implementation steps include:

[0018] Step S31: Use a 1D convolutional neural network as the encoder of the prototype network to map the 310-dimensional DE features into a low-dimensional embedding representation;

[0019] Step S32: Calculate the feature attention weights;

[0020] Step S33: Calculate the channel-band attention weights.

[0021] Step S4: Calculate the prototype representation, fuse the feature attention weights and channel-band attention weights to obtain the final weighted distance.

[0022] Preferably, the specific implementation process includes:

[0023] Step S41: Introduce gradient fusion weights, and use weighted summation to fuse projection weights, gradient fusion weights, and feature attention weights to obtain the final fusion weights;

[0024] Step S42: Calculate the prototype vector for each classification category based on the support set embedding and attention weights;

[0025] Step S43: Design multiple loss functions.

[0026] Step 5: Introduce the Grad-CAM visualization mechanism, calculate the average Grad-CAM heatmap, the average feature attention weight, the average channel-band attention weight, and the average fusion weight, and analyze the consistency of the weight distribution.

[0027] Step 6: Input the fused representation into the classifier, calculate the classification loss based on the prediction score, and output the cross-time EEG classification results.

[0028] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0029] (1) Strong cross-time domain adaptability: By using conditional MMD loss (focusing on the distribution of similar samples) and reconstruction loss (preserving the original features), the distribution shift of EEG across time periods is effectively mitigated, and the domain generalization is improved;

[0030] (2) Multi-dimensional attention synergistic optimization: It integrates "feature attention + channel-band attention + gradient attention" to capture key information in the feature dimension and task response specificity of the EEG channel-band, which is more accurate than a single attention mechanism;

[0031] (3) High interpretability: Based on Grad-CAM, the attention weights and key brain regions / bands are visualized, and the rationality of the decision is verified by cosine similarity and entropy distribution, thus solving the "black box" problem of deep learning;

[0032] (4) Robustness and practicality: K-shot few-sample sampling (adapted to scenarios with limited labeled samples), gradient truncation (avoiding gradient explosion), and data augmentation (anti-noise interference) are adopted. It is easy to reproduce on public datasets and can be directly transferred to actual EEG classification scenarios. Attached Figure Description

[0033] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0034] Figure 1 This is a framework diagram of a hybrid attention prototype network EEG classification method that integrates feature weights, provided by an embodiment of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1

[0037] This invention provides a technical solution: a hybrid attention prototype network EEG classification method that integrates feature weights, comprising the following steps:

[0038] Step S1: Collect EEG data from multiple subjects using specific tasks at different time periods.

[0039] In this embodiment, EEG data of multiple subjects were collected under specific tasks (such as viewing designated images or videos) at different time periods. The specific requirements are as follows:

[0040] Dataset: This invention uses the publicly available EEG dataset based on SEED-IV. This dataset contains task-related EEG data from multiple subjects in three time periods. Each subject completes a specified task in three time periods, and each time period contains multiple valid EEG trials (trials with more than 15% artifacts are removed).

[0041] Data Acquisition Equipment: This invention employs a 62-channel EEG cap conforming to the international 10-20 system, with reference electrodes. Original sampling rate Sampling rate after downsampling .

[0042] Classification categories: Supports multi-class EEG classification; the number of classification categories in the dataset parameters. (Corresponding to the 4 types of EEG data in the SEED-IV dataset), number of participants Number of time periods .

[0043] Data storage: EEG data and corresponding classification labels are stored separately as follows: (Differential entropy feature file) and (Label file), where "subject" is the subject number and "session" is the time period number.

[0044] Step S2: Preprocess the collected raw EEG data, extract differential entropy (DE) features, and generate support sets and query sets by constructing datasets for the source and target domains and standardizing and sampling the extracted features.

[0045] In this embodiment, the specific implementation process is as follows:

[0046] Step S21: Perform preprocessing operations such as filtering, denoising, and zero-mean normalization on the original EEG signal.

[0047] For example, a 5th-order Butterworth bandpass filter (frequency band 8-70Hz) is used to remove low-frequency drift and high-frequency noise, a 60Hz notch filter is used to remove power frequency interference, and independent component analysis (ICA) is used to remove artifacts of electrooculography (EOG) and electromyography (EMG), and finally the sampling rate is downsampled to 200Hz.

[0048] Zero-mean standardization: Features in the source domain (training period) and target domain (testing period) are standardized separately to avoid differences in data distribution within the domains. The formula is:

[0049]

[0050] in, For standardized features, The original feature matrix, The characteristic mean, The characteristic standard deviation, To avoid smoothing terms that divide by zero.

[0051] Step S22: Extract differential entropy (DE) features from the differential entropy feature file of the public dataset;

[0052] For example, the energy distribution of EEG signals in different frequency bands reflects emotion-related brain activity. Specifically: Frequency band division: delta (1-4Hz), theta (4-8Hz), alpha (8-14Hz), beta (14-31Hz), and gamma (31-50Hz), totaling 5 frequency bands; Channel-frequency band combination: Extracting DE features from 62 EEG channels × 5 frequency bands to form a feature matrix with a dimension of "sample number × 310" (62 × 5 = 310), denoted as... (n is the number of samples).

[0053] Step S23: Construct a source domain-target domain training / validation / test split, and use balanced sampling to ensure that the number of samples for each classification category is consistent;

[0054] For example, splitting a dataset includes:

[0055] For the source domain: it is split into a training set (for model pre-training) and a validation set (for hyperparameter tuning) in an 8:2 ratio.

[0056] Target domain: split into a fine-tuning set (for model fine-tuning) and a test set (for final evaluation) in a 2:8 ratio.

[0057] Sampling function: using (Fixed ratio sampling) or _ (Random sampling) function, drawing from each subset One support sample (for computing the prototype) and There are 10 query samples (used for classification), among which , (Default parameter, can be adjusted via parser) Ensure that the number of supported samples for each category is equal to the number of query samples.

[0058] Step S3: Input the preprocessed features into the prototype network, extract the embedding representations of the source / target domain, and calculate the feature weights and channel-band weights through a dual attention layer.

[0059] In this embodiment, the specific process is as follows:

[0060] Step S31: Use a 1D convolutional neural network (CNN) as the encoder of the prototype network to map the 310-dimensional DE features into a low-dimensional embedding representation.

[0061] For example, the prototype network encoder uses a 1DCNN to map 310-dimensional DE features into 128-dimensional embeddings, that is, the input... First, expand it to a 3D tensor of "sample number × 1 × 310" (adapted to 1D convolution), and output a 128-dimensional embedding vector. (Source domain embedding) Target domain embedding Supports embedded sets Query set embedding Embedded vectors Represented as:

[0062]

[0063] in, For DE features, This represents the DE feature after dimensional expansion.

[0064] Step S32: Calculate the feature attention weights.

[0065] For example, this invention designs feature attention and channel-band attention to mine EEG classification-related information from the feature dimension and channel-band dimension, respectively, and performs a linear transformation on the mean of the support set embedding. Normalization highlights the contribution of key feature dimensions, i.e., calculating feature attention weights. The calculation formula is:

[0066]

[0067] in, For linear layer weights, To support the mean vector of set embedding, (Consistent with the embedding dimension).

[0068] Step S33: Calculate the channel-band attention weights.

[0069] For example, multi-head attention is used to capture the dependencies between channels and frequency bands, and then a linear layer is used to map these dependencies into a 62×5 matrix (corresponding to 62 channels × 5 frequency bands), which represents the channel-frequency band attention weights. The calculation is performed using the following formula:

[0070] in, For the linear layer weights (310 = 62 × 5) For an 8-head attention mechanism (code parameters), .

[0071] Step S4: Fuse the feature attention weights and channel-band attention weights to obtain the final weighted distance.

[0072] In this embodiment, gradient fusion is used to optimize the dual attention weights, and the class-conditional MMD loss, reconstruction loss and classification loss are combined to ensure cross-time domain adaptability and recognition accuracy.

[0073] For example, the specific implementation process is as follows:

[0074] Step S41: Introduce gradient fusion weights, and use weighted summation to fuse projection weights, gradient fusion weights, and feature attention weights to obtain the final fusion weights.

[0075] For example, gradient fusion weights are introduced in this invention. The attention allocation is optimized by utilizing the gradient information of the weighted distance and fusing the projection weights. Gradient fusion weights and feature attention weights Channel-band attention Flattened into a 310-dimensional vector, and passed through a linear layer Projecting to 128 dimensions yields Then calculate the weighted distance sum with respect to the projection weights. The gradient is truncated to [-5, 5] using the clamp function to avoid gradient explosion, and the gradient fusion weights are calculated. The calculation formula is:

[0076]

[0077] in, To obtain the partial derivative, In order to reach a settlement, This is the weighted distance. If gradient calculation fails (e.g.) If None is returned, then the gradient fusion weights are... .

[0078] Weighted summation and fusion projection weights Gradient fusion weights and feature attention weights Three types of weights are used to obtain the fusion weight. The calculation formula is:

[0079]

[0080] in, , As weighting coefficients, in this invention The values ​​are 0.5, 0.3, and 0.01, respectively.

[0081] Step S42: Calculate the prototype vector for each classification category based on the support set embedding and attention weights.

[0082] For example, calculate the prototype vector (category center) for each classification category. The calculation formula is:

[0083]

[0084] in, This is the prototype of category c. For the embedding of the k-th supporting sample of category c, To support the attention score of the set-query set, the attention score is calculated using the following formula:

[0085]

[0086] in, , These are the transformation matrices for the support set and query set of category c, respectively. For feature dimensions.

[0087] Step S43: Design multiple loss functions.

[0088] For example, by jointly optimizing the model using classification loss, class-conditional MMD loss, and reconstruction loss, cross-time domain adaptation and recognition accuracy can be ensured:

[0089] Weighted distance metric: Calculates the squared weighted Euclidean distance between the query sample embedding and the prototype based on fusion weights. The calculation formula is:

[0090]

[0091] in, Let k be the fusion weight. , These are the k-th dimension elements of the query embedding and prototype, respectively.

[0092] Classification loss Calculate the cross-entropy loss using negative weighted distance as logits. The calculation formula is:

[0093]

[0094] in, To query the unique hot tags of the sample, is the weighted distance of the i-th query sample.

[0095] Class-Conditional MMD Loss: Measures the difference in the distribution of embeddings of similar samples between the source and target domains. The calculation formula is as follows:

[0096]

[0097] Where c is the number of categories, For multi-scale kernel parameters, The kernel function is defined as the scale parameter:

[0098]

[0099]

[0100] in, This represents the mean kernel similarity of samples of the same type in the source domain. This represents the mean kernel similarity among similar samples in the target domain. This represents the mean kernel similarity between similar samples in the source and target domains. These represent the source / target domain distributions for category c. This represents the sample mean (empirical expectation).

[0101] Reconstruction loss To encourage the dispersion of channel-band attention weights and avoid overfitting to a single channel / band, the calculation formula is as follows:

[0102]

[0103] in, This represents the batch average. For channel-band attention, the flattened vector is... This indicates that the variance is calculated along the sample dimension.

[0104] Total loss function: weighted fusion of three types of loss, calculated using the following formula:

[0105]

[0106] in, The loss weights for MMD. To reconstruct the loss weight hyperparameters, in this invention... Take 0.1.

[0107] Step 5: Introduce the Grad-CAM visualization mechanism to calculate the average Grad-CAM heatmap. Weighted average, Average and The average weight is used to analyze the consistency of the weight distribution.

[0108] In this embodiment, a Grad-CAM (Gradient-Weighted Class Activation Mapping) mechanism is introduced to visualize the attention weight allocation and key brain regions / bands, verifying the consistency of the weight distribution. Gradient capture is performed with classification score as the target, obtaining the encoder's intermediate layer gradients and attention outputs, followed by heatmap generation. The generation formula is:

[0109]

[0110] in, For feature maps Find the gradient. This is the key feature map output by the model encoder.

[0111] Table 1: Model performance on the SEED-IV dataset

[0112]

[0113] As can be seen from the results in Table 1, the accuracy of the present invention is the lowest in the case of 1-shot and the highest in the case of 7-shot. The more shots a given invention has, the more accurate the prototype generation will be.

[0114] Step S6: Finally, input the fused representation into the classifier, calculate the classification loss based on the prediction score, and output the cross-time EEG classification results.

[0115] Example 2

[0116] In this embodiment, the present invention provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The memory stores the computer program; the processor executes a hybrid attention prototype network EEG classification method with fused feature weights as described in Embodiment 1.

[0117] Example 3

[0118] In this embodiment, the present invention further provides a readable storage medium containing a computer program; when the processing unit runs, the computer program is able to complete all the steps of the "a hybrid attention prototype network EEG classification method with fused feature weights" in Embodiment 1.

[0119] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0120] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A hybrid attention prototype network EEG classification method that integrates feature weights, characterized in that: The following steps are included: Step S1: Collect EEG data from multiple subjects using specific tasks at different time periods; Step S2: Preprocess the collected raw EEG data, extract differential entropy features, and generate support sets and query sets by constructing datasets for the source and target domains and standardizing and sampling the extracted features. Step S3: Input the preprocessed features into the prototype network, extract the embedding representations of the source / target domain, and calculate the feature weights and channel-band weights through a dual attention layer; Step S4: Calculate the prototype representation, fuse the feature attention weights and channel-band attention weights to obtain the final weighted distance; Step 5: Introduce the Grad-CAM visualization mechanism, calculate the average Grad-CAM heatmap, the average feature attention weight, the average channel-band attention weight, and the average fusion weight, and analyze the consistency of the weight distribution; Step 6: Input the fused representation into the classifier, calculate the classification loss based on the prediction score, and output the cross-time EEG classification results; The implementation process of step S4 includes: Step S41: Introduce gradient fusion weights, and use weighted summation to fuse projection weights, gradient fusion weights, and feature attention weights to obtain the final fusion weights; Step S42: Calculate the prototype vector for each classification category based on the support set embedding and attention weights; Step S43: Design multiple loss functions; Step S41 also includes: flattening the channel-band attention into a 310-dimensional vector, projecting it to 128 dimensions through a linear layer to obtain the projection weights; calculating the gradient of the weighted distance sum with respect to the projection weights, and truncating it to [-5,5] using the clamp function to avoid gradient explosion; then calculating the gradient fusion weights; finally, using weighted summation to fuse the projection weights, gradient fusion weights, and feature attention weights to obtain the fused weights. Step S43 involves designing multiple loss functions, including: calculating the weighted squared Euclidean distance between the query sample embedding and the prototype based on the fusion weights; calculating the cross-entropy loss using the negative weighted distance as logits; calculating the class-conditional MMD loss, which measures the distribution difference between the embeddings of similar samples in the source and target domains; encouraging the dispersion of channel-band attention weights and calculating the reconstruction loss; and obtaining the total loss function by weighting the fusion cross-entropy loss, class-conditional MMD loss, and reconstruction loss.

2. The EEG classification method using a hybrid attention prototype network with fused feature weights according to claim 1, characterized in that: The steps to implement step S2 include: Step S21: Perform preprocessing operations such as filtering, denoising, and zero-mean normalization on the original EEG signal; Step S22: Extract differential entropy features from the differential entropy feature file of the public dataset; Step S23: Construct a source domain-target domain training / validation / test split, and use balanced sampling to keep the number of samples for each classification category consistent.

3. The EEG classification method using a hybrid attention prototype network with fused feature weights according to claim 2, characterized in that: The zero-mean standardization formula in step S21 is: ； in, For standardized features, The original feature matrix, The characteristic mean, The characteristic standard deviation, This is a smoothing term.

4. The EEG classification method using a hybrid attention prototype network with fused feature weights according to claim 3, characterized in that: Step S22 involves extracting differential entropy features by loading a file from a public EEG dataset, extracting DE features from 62 EEG channels × 5 frequency bands, and forming a feature matrix with a dimension of "number of samples × 310".

5. The EEG classification method using a hybrid attention prototype network with fused feature weights according to claim 4, characterized in that: The steps to implement step S3 include: Step S31: Use a 1D convolutional neural network as the encoder of the prototype network to map the 310-dimensional DE features into a low-dimensional embedding representation; Step S32: Calculate the feature attention weights; Step S33: Calculate the channel-band attention weights.

6. The EEG classification method of a hybrid attention prototype network with fused feature weights according to claim 5, characterized in that: Step S32, which involves calculating the feature attention weights, includes: Design feature attention and channel-band attention; EEG classification-related information is mined from both the feature dimension and the channel-band dimension. Perform a linear transformation on the mean of the support set embeddings and Normalize and calculate feature attention weights.

7. The EEG classification method using a hybrid attention prototype network with fused feature weights according to claim 6, characterized in that: Step S33, which involves calculating the channel-band attention weights, includes: Capture channel-band dependencies through multi-head attention; Then, it is mapped to a 62×5 matrix through a linear layer, corresponding to 62 channels × 5 frequency bands; Calculate channel-band attention weights.

Images