A multi-view feature based modular adaptive electroencephalogram (EEG) classification method

Abstract

The application provides a module adaptive electroencephalogram (EEG) classification method based on multi-view features. First, a multi-domain feature extraction network is constructed, which can separate multiple view features such as time domain, frequency domain and space domain and realize EEG signal classification. Second, a modular transfer adaptation strategy is used to realize network parameter fine-tuning. The transfer quantification is used to quantify the effectiveness of each view feature in cross-subject transfer, and the clustering analysis based on the Gaussian mixture model is used to realize adaptive filtering of the transfer module. Through the cooperative transfer of multiple view features and the modular adaptive adjustment, the application can realize the effective transmission of domain-invariant features and the dynamic adaptation of domain-specific features, significantly improving the robustness and classification accuracy of the network model in the cross-subject EEG signal classification task.

Description

Technical Field

[0001] This invention belongs to the field of medical signal processing technology, specifically relating to a modular adaptive EEG signal classification method based on multi-view features. Background Technology

[0002] In 1999, the first International Brain-Computer Interface Conference was held in New York, where the concept of Brain-Computer Interface (BCI) was proposed. Utilizing BCI technology, the brain, as the primary means of communication and control between the user and the outside world, can better exert its thought control capabilities. Electroencephalography (EEG), as an important bioelectrical signal recording brain neural activity, can reflect an individual's cognitive state, emotional changes, and the brain's functional activities in different tasks and environments. Precise analysis and decoding of EEG signals are key to a deeper understanding of the brain's working mechanisms, revealing the principles of neurological diseases, and promoting the development of BCI technology. Among these technologies, non-invasive electroencephalography (EEG) has become a cornerstone technology for BCI research due to its non-invasiveness, high real-time performance, low cost, and rich application ecosystem. Despite its limitations, its universality in medical and consumer fields, as well as its potential for synergy with other technologies, ensures that it will remain a core driving force for BCI development in the foreseeable future.

[0003] Electroencephalogram (EEG) signals are essentially a macroscopic representation of brain neural activity, but the raw signals themselves cannot directly reflect the user's intentions. Researchers need to specifically extract features from the EEG signals of motor imagery and classify and identify the extracted features to convert the EEG signals into executable commands. However, the low signal-to-noise ratio and non-stationarity of EEG signals make accurate interpretation quite difficult. In particular, significant differences in brain structure, functional connectivity patterns, and cognitive task execution strategies among individuals lead to obvious individual variations in EEG signals, further increasing the difficulty of interpreting them.

[0004] Current mainstream EEG signal classification methods are primarily based on two assumptions: first, building a general classification model based on multi-subject data, assuming that the EEG characteristics of different individuals are isomorphic; second, training a dedicated model for each individual. However, due to significant individual differences in EEG signals, general models suffer from feature mismatch when applied to new individuals. For example, when different subjects perform the same motor imagery task, there are significant differences in the activation intensity and phase synchronization of their μ-rhythm (8-12Hz) and β-rhythm (18-26Hz) EEG signals. This makes it difficult for general models to accurately capture individual-specific patterns, severely impacting their accuracy in clinical diagnosis and their practical value in personalized brain-computer interface systems. Even more critically, the high cost of EEG signal acquisition results in extremely limited usable data for a single subject. Conventional deep learning methods are prone to overfitting under small sample conditions, causing a sharp decline in the model's generalization performance on new data, thus hindering the practical application of related technologies.

[0005] To address these challenges, researchers have explored techniques such as domain adaptation, few-shot learning, and meta-learning, aiming to alleviate data scarcity through cross-individual knowledge transfer. Traditional transfer learning methods generally employ model fine-tuning or feature reuse strategies. However, these methods use a uniform transfer mechanism to handle different target individuals, failing to effectively distinguish between domain-invariant features (such as time-frequency response patterns in motion imagery tasks) and domain-specific features (such as signal shifts caused by individual physiological characteristics), resulting in low efficiency in key feature transfer. As a mainstream domain adaptation method, distribution alignment-based methods achieve knowledge transfer by minimizing the difference in feature distribution between the source and target domains. However, they only align at the overall distribution level, ignoring the differentiated transfer value of different feature subsets and failing to effectively identify the moderating effect of individual-specific features. Existing methods often employ single-view feature alignment strategies, utilizing only a single type of feature (such as spatial domain features) for transfer learning. This fails to fully exploit the complementarity between multi-view features (time-frequency domain, spatial domain, phase features, etc.) and lacks a quantitative evaluation mechanism for the quality of transferred features, leading to a blind transfer process.

[0006] Overall, even though existing methods can utilize the spatiotemporal characteristics of EEG signals to complete classification tasks, they cannot be effectively applied to the EEG signals of different individuals. In other words, these methods cannot fundamentally alleviate the challenges of classifying brain EEG signals. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention provides a modular adaptive EEG signal classification method based on multi-view features. First, addressing the problem of insufficient feature complementarity, a multi-view feature extraction network is constructed, capable of separating multiple view features in the time, frequency, and spatial domains and achieving EEG signal classification. Second, a modular transfer adaptation strategy is employed to fine-tune network parameters. Transfer scores are used to quantify the effectiveness of each view feature in cross-individual transfer, and clustering analysis based on a Gaussian mixture model is used to achieve adaptive selection of transfer modules. This invention, through the collaborative transfer of multiple view features and modular adaptive adjustment, enables the effective transfer of domain-invariant features and the dynamic adaptation of domain-specific features, significantly improving the robustness and classification accuracy of the network model in EEG signal classification tasks across subject scenarios.

[0008] A modular adaptive EEG signal classification method based on multi-view features, characterized by the following steps: Step 1, EEG signal preprocessing: For existing EEG signal data containing different subjects performing different imagery tasks, the EEG signals of the corresponding task categories are extracted and filtered according to the labels. The mean baseline correction method is used to correct the baseline of the filtered data. The ICA method is used to remove eye movement artifacts from the baseline-corrected data, and then the data is normalized. The target domain dataset is composed of all EEG signals of a single subject, and the source domain dataset is composed of all EEG signals of other subjects. The target domain dataset is divided into training set and validation set according to the number of EEG signal samples in a 4:1 ratio. Step 2, Network Training: The multi-view feature extraction network is trained using the source domain dataset after preprocessing in Step 1 as input and the EEG signal classification results as output. The multi-view feature extraction network includes a frequency domain feature extraction module and a spatiotemporal feature extraction module deployed in parallel, as well as a classifier module. The frequency domain feature extraction module extracts the frequency domain feature map of the input data, and the spatiotemporal feature extraction module extracts the temporal domain feature map and the spatial domain feature map of the input data. The feature maps from different domains are fused and input into the classifier module. The features are mapped to the target class space through a fully connected classification head, and the probability distribution is calculated using the Softmax function to obtain the classification result. The specific processing procedure of the frequency domain feature extraction module is as follows: First, input the electroencephalogram (EEG) signal. ,right X Perform frequency band filtering to obtain filtered data. ,in, C Indicates the number of channels. T Indicates the number of sampling points. N b This indicates the number of frequency band filters; then, the differential entropy features of the filtered data are extracted. and power spectral density characteristics Then, the features are spliced ​​together to obtain the spliced ​​features. ,in, N s Indicates the number of segments in the time dimension. N psd This represents the feature length of the single-band power spectral density (PSD); then, the stitched feature F is input into the linear projection layer, according to... Calculate the feature data ,in, , D This represents the feature dimension generated after linear projection; finally, the feature data... z lp After adding position encoding, the input is a feature fusion network consisting of several Transformer encoders connected in series, resulting in the final frequency domain feature map; The spatiotemporal feature extraction module includes a multi-scale convolution module, a channel attention module, and a spatial convolution module. The multi-scale convolution module uses several 1D convolution kernels of varying sizes to perform independent convolution operations on the input EEG signal. The feature maps obtained from each convolution operation are summed to obtain the temporal feature map of the EEG signal. The temporal feature map is then processed by the channel attention module to obtain a weighted feature map. This weighted feature map undergoes batch normalization and a ReLU activation function before being fed into the spatial convolution module. The spatial convolution module uses a 3×1×1 convolution kernel to perform convolution operations on the input feature map, followed by batch normalization and a ReLU activation function to obtain the spatial feature map of the EEG signal. The specific implementation process of fusing features from different domains in the multi-view feature extraction network is as follows: First, the feature maps of different domains are decoupled and unfolded along their sample dimensions, then concatenated along the channel dimension and aligned along the spatial dimension to obtain the concatenated multi-domain features; then, the concatenated multi-domain features are divided into sliding window blocks through an image patch embedding layer, and the feature vectors in each window are mapped to the latent space through a learnable linear projection to generate an embedding vector sequence with positional encoding information, and a class identifier is introduced as a trainable parameter and inserted into the beginning of the sequence; finally, multi-domain feature fusion is achieved through a stacked Transformer encoder. Step 3, Network Fine-tuning: The multi-view feature extraction network trained in Step 2 is adjusted using a modular transfer adaptation strategy; Step 4, EEG signal classification: Input the EEG signal dataset to be classified into the adjusted multi-view feature extraction network to obtain the classification results.

[0009] Specifically, the filtering process includes first using a Butterworth bandpass filter from 0.5Hz to 100Hz to filter out high-frequency noise interference in the EEG signal, and then using a 50Hz notch filter to suppress power frequency noise in the EEG signal; the data normalization process uses z-score normalization.

[0010] Specifically, the differential entropy feature extraction method is as follows: for continuous random variables X If its probability density function is f ( x Its differential entropy characteristics are: ; The power spectral density feature extraction method is as follows: First, the original signal... x ( n Divide into uniform segments, with the number of segments being... L Each segment is [length] M Overlapping sections are set between segments, with the number of overlap points... Pick M / 2, apply a window function to each signal segment. The window function value smoothly decays to zero at the segment endpoints; then, a discrete Fourier transform is performed on each windowed signal segment, and the power spectral density estimate is obtained according to Parseval's theorem. (1) in, Indicates the first m segment signal x m ( n At frequency point k The power spectral density estimate at that location, , The complex exponential basis used in calculating the Fourier transform, U The normalization factor is calculated using the following formula: ; Finally, according to Calculated signal x ( n The final power spectral density estimate.

[0011] Specifically, the convolution operation is a same convolution; The channel attention module first uses global average pooling to compress the feature map output by the multi-scale convolution module by channel to obtain the global features of each channel. Then, it uses two fully connected layers of the same size to learn the relationship between channels, uses the non-linear activation function Sigmoid to obtain the channel attention weights, and multiplies the channel attention weights with the features of the corresponding channels to output the weighted feature map.

[0012] Specifically, the following total loss function is used in the network training: (2) in, Indicates the total network loss. Indicates domain loss, Indicates classification loss; The aforementioned domain loss The calculation formula is as follows: (3) in, This represents the signal samples in the source domain dataset. This represents the signal samples in the target domain dataset. Indicates the number of feature maps. M =3, representing the frequency domain feature map, time domain feature map, and spatial domain feature map, respectively. i, j This represents the sequence number of the signal sample in the source domain dataset. p , q This represents the sequence number of the signal sample in the target domain dataset. The multi-kernel function representing two signals is calculated using the following formula: (4) in, It is the first u A signal x and y kernel function, The corresponding coefficients satisfy... , k The number of kernel functions; The classification loss The calculation formula is as follows: (5) in, Indicates the first i One signal sample, Indicates the first i The label of each signal sample, The total number of samples, This indicates the multi-view feature extraction network. This represents the calculation of the cross-entropy loss function; During training, the best network model is obtained by minimizing the total loss function using the Adam optimization algorithm.

[0013] Specifically, the process of adjusting the network using the modular transfer adaptation strategy is as follows: First, calculate the transfer score vectors of the feature extraction modules corresponding to different views. Then, using the transfer score vectors as samples, all transfer score vectors form a new sample set. Use a Gaussian mixture model to perform cluster analysis on the samples in this set, dividing them into two classes: low-mean clustered samples are designated as the frozen class, and other samples are designated as the fine-tuning class. Finally, input the target domain training dataset into the multi-view feature extraction network for retraining. During training, keep the parameters of the feature extraction modules for the views corresponding to the frozen class samples unchanged, and adjust the parameters of the feature acquisition modules for the views corresponding to the fine-tuning class samples to complete the network fine-tuning.

[0014] Specifically, the Transformer encoder is a composite computational unit comprising a multi-head attention mechanism, layer normalization, residual connections, and a feedforward network. It establishes long-range dependencies between cross-domain features through a multi-head self-attention mechanism and quantifies the correlation strength between frequency-domain and time-domain features using a cross-attention weight matrix. Each encoder stage achieves multi-domain feature fusion through cascaded multi-layer nonlinear transformations. Specifically, the transfer score vector is composed of the distribution difference scores between the source and target domains. Discrimination score of target domain The composition, and its specific calculation process are as follows: Step (1): Calculate the following formula: m Feature map The signal sample distribution difference score between the source domain dataset and the target domain dataset is as follows. : (6) in, This represents the signal samples in the source domain dataset. This represents the signal samples in the target domain dataset. i, j This represents the sequence number of the signal sample in the source domain dataset. p , q This represents the sequence number of the signal sample in the target domain dataset. m =1,2,…, M , (·,·) is a positive definite kernel function, using a Gaussian kernel function. , x and y This represents two signals in the input space. It is a bandwidth parameter; Step (2): Calculate the following formulas respectively. m Feature map The average inter-class distance between different class signal samples in the target domain dataset below and average intra-class distance : (7) (8) in, express distance, C This represents the total number of classes of signal samples in the target domain dataset. This represents calculating the central feature vector of all signal samples in a certain category. Indicate category All signal samples, Indicate category All signal samples, Indicate category c All signal samples, Indicates the category All samples Input to extract the first m The feature vector set of all samples of this category obtained after the feature extraction module of each feature map. Indicates the category All samples Input to extract the first m The feature vector set of all samples of this category obtained after the feature extraction module of each feature map. Indicates the category c All samples Input to extract the first m The feature vector set of all samples of this category obtained after the feature extraction module of each feature map. c 1. c 2. c The table lists the category numbers respectively. Indicate category c The total number of signal samples contained in i Indicates the signal sample number. Indicate category c The first in i One signal sample, Indicates the category c The i signal samples Input to extract the first m The feature extraction module of each feature map obtains the features of the sample; This represents calculating the central feature vector of all signal samples in a certain category. For category A... c All signal samples , its first m The formula for calculating the central eigenvector under each feature map is: (9) Step (3): Calculate the following formula: m A view Discriminant scores of the target domain dataset below : (10) Step (4): with As the first m The transfer score vector of the feature extraction module corresponding to each view.

[0015] Specifically, the process of clustering the migration score vector using a Gaussian mixture model is as follows: the optimal mean is found by maximizing the log-likelihood function. Covariance Matrix This clusters the transfer score vectors into two categories: frozen and fine-tuned, as expressed by the formula: (11) in, It represents the set of migration score vectors across all views. , This represents the clustering result of all migration score vectors. If the output is 0, the input is considered to be in the frozen category; if the output is 1, the input is considered to be in the fine-tuning category. It is clustering c The mean vector, It is the corresponding covariance matrix. It is clustering c The probability, It is the first m The transfer score vector of the feature extraction module corresponding to each view; c =1 or 2 represent the freeze class and the fine-tuning class, respectively.

[0016] The beneficial effects of this invention are as follows: Because a multi-view feature extraction network is constructed, it can separate multiple view features such as time domain, frequency domain, and spatial domain, and achieve EEG signal classification; because the network adopts a modular design, it is beneficial to adaptively adjust parameters in each module to obtain a better network model, thereby obtaining better classification results; because a transfer score calculation method based on multi-view features is adopted, it can better express the multi-domain distribution differences of EEG signals, and then determine the transfer path based on the differences, enabling effective transfer of domain-invariant features and dynamic adaptation of domain-specific features, significantly improving the robustness and classification accuracy of the network model in EEG signal classification tasks in cross-subject scenarios. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the basic process of the modular adaptive EEG signal classification method based on multi-view features of the present invention; Figure 2 This is a schematic diagram illustrating the network fine-tuning process using a modular migration adaptation strategy in this invention. Detailed Implementation

[0018] The present invention will be further described below with reference to the accompanying drawings and embodiments. The present invention includes, but is not limited to, the following embodiments.

[0019] The operating environment for this embodiment is as follows: The server used for model training is configured with two GeForce GTX 3090 GPUs with 24GB of video memory each, 256GB of RAM, and the operating system is Ubuntu 22.04LTS. PyTorch 2.5.0 is used as the deep learning framework, Python 3.8 as the programming language, and VS Code as the integrated development environment for related experiments.

[0020] The experiment used the Brain-Computer Interface Competition dataset (BCI Competition IV 2a / 2b). The BCI Competition IV 2a dataset includes EEG data from 9 participants on motor imagery. The cue-based BCI paradigm includes four different motor imagery tasks: left-hand, right-hand, bilateral foot, and tongue motor imagery. Each participant performed two experiments on different dates, divided into sessions. Each session consisted of 6 runs with appropriate rest periods between runs. One run consisted of 48 trials (12 for each of the four imagery categories), for a total of 288 trials per session. At the beginning of each session, participants underwent 5 minutes of EEG data acquisition to assess the impact of EOG (electroophthalmography) and to address eye movement artifacts during subsequent preprocessing. The participants in the BCI Competition IV 2b dataset were consistent with those in the BCI Competition IV 2a dataset, and they performed left-hand and right-hand motor imagery tasks without feedback at different times. The experiment was divided into sessions, each containing 6 runs. Each run consisted of 10 trials of two types of tasks, for a total of 120 trials per session. Each participant had 120 sets of data for each type of motor imagery task. EEG signals were acquired using three bipolar electrodes at a sampling frequency of 250 Hz at locations C3, C2, and C4. The experimental paradigm and other experimental settings were consistent with BCI Competition IV 2a.

[0021] like Figure 1 As shown, the specific implementation process of the present invention is as follows; 1. Preprocessing of EEG signals For existing EEG signal data containing different subjects performing different imagery tasks, EEG signals of the corresponding task categories were extracted and filtered according to labels. Baseline correction was performed on the filtered data using the mean baseline correction method (i.e., all subjects' EEG signals were referenced to the same zero point to avoid adverse effects on subsequent analysis due to differences in baseline levels between two groups of subjects). Eye movement artifacts (EOG data) were removed from the baseline-corrected data using the ICA method. To eliminate differences in amplitude and characteristics between different EEG signals in each dataset and ensure comparability, the EEG signals were normalized. Then, the target domain dataset was constructed using all EEG signals from a single subject, and the source domain dataset was constructed using all EEG signals from other subjects. The target domain dataset was divided into training and validation sets at a 4:1 ratio based on the number of EEG signal samples.

[0022] Specifically, the filtering process is as follows: First, a Butterworth bandpass filter from 0.5Hz to 100Hz is used for bandpass filtering to remove high-frequency noise interference in the EEG signal. Then, a 50Hz notch filter is used to suppress power frequency noise in the EEG signal.

[0023] Specifically, data normalization employs z-score normalization. z-score normalization works by subtracting the mean from each sample value of the original signal and then dividing by its standard deviation, resulting in standardized data with a mean of 0 and a standard deviation of 1. This method is suitable for data with an approximately normal distribution. In most EEG signal analyses, if the data does not exhibit a significant skewness, z-score normalization effectively ensures comparability of data under different conditions. The z-score calculation formula is: (12) in, x i Indicates input data, x 0 indicates the standardized output. μ and σ These represent the mean and variance of the input data, respectively.

[0024] 2. Network training To address the limited generalization ability of models in EEG signal classification tasks due to differences across subjects and scenarios, this invention first constructs a modularly designed multi-view feature extraction network, such as... Figure 1As shown, it mainly includes a frequency domain feature extraction module and a spatiotemporal feature extraction module deployed in parallel, as well as a classifier module. The frequency domain feature extraction module extracts the frequency domain feature map of the input data, and the spatiotemporal feature extraction module extracts the temporal domain feature map and the spatial domain feature map of the input data. The feature maps of different domains are fused and input into the classifier module. The features are mapped to the target class space through a fully connected classification head, and the probability distribution is calculated using the Softmax function to obtain the classification result.

[0025] Specifically, such as Figure 1 As shown, the specific processing procedure of the frequency domain feature extraction module is as follows: First, input the EEG signal. ,right X Perform frequency band filtering to obtain filtered data. ,in, C Indicates the number of channels. T Indicates the number of sampling points. N b This indicates the number of frequency band filters; then, the differential entropy features of the filtered data are extracted. and power spectral density characteristics Then, the features are spliced ​​together to obtain the spliced ​​features. ,in, N s Indicates the number of segments in the time dimension. N psd This represents the feature length of the single-band power spectral density (PSD); then, the stitched feature F is input into the linear projection layer, according to... Calculate the feature data ,in, , D The feature dimensions generated after representational projection; finally, the feature data... z lp After adding position encoding, the input is a feature fusion network consisting of several Transformer encoders connected in series, resulting in the final frequency domain feature map.

[0026] The differential entropy feature extraction method is as follows: for continuous random variables X If its probability density function is f ( x Its differential entropy (DE) characteristics are: .

[0027] The power spectral density feature extraction method is as follows: First, the original signal... x ( n Divide into even segments, each segment being [length missing]. M Overlapping sections are set between segments, with the number of overlap points... Pick M / 2 (which can usually be adjusted according to the actual situation), signal segmentation can make the statistical characteristics of each segment of the signal more stable, which is beneficial for subsequent accurate estimation. Let the length of the original signal be . N , the number of segments that can be L for: (13) Apply a window function to each signal segment (Such as the Hanning window function), the window function value smoothly decays to zero at the segment endpoints, suppressing spectral leakage caused by abrupt changes at signal truncation edges. Windowing can reduce the impact of edge discontinuities on the spectrum. Taking the Hanning window function as an example... The first window added The segment signal is:

[0028] in, n For sampling points, This is the signal after applying a window function.

[0029] Then, discrete Fourier transforms are performed on each windowed signal segment, and the power spectral density estimate (i.e., periodogram) is obtained according to Parseval's theorem. (14) in, Indicates the first m segment signal x m ( n At frequency point k Estimated power spectral density at a location (period plot). , The complex exponential basis used in calculating the Fourier transform, U The normalization factor is calculated using the following formula: ; Finally, regarding the frequency point k Average all segments of the periodic chart, i.e., according to Calculated signal x ( n The final power spectral density estimate.

[0030] The Transformer encoder is N A series of cascaded composite computational units, including multi-head attention mechanisms, layer normalization, residual connections, and feedforward networks, establish long-range dependencies between cross-domain features through multi-head self-attention mechanisms. Cross-attention weight matrices quantify the correlation strength between frequency-domain and time-domain features. Each encoder stage achieves multi-domain feature fusion through cascaded multi-layer nonlinear transformations. The computation process can be represented by the following formula: (15) (16) in, l This indicates the layer number of the encoder. For the encoder's layer number... l layer, Indicates the first l The input feature data of the layer encoder, Indicates the first l The output feature data of the layer encoder , Indicates position code, This is the intermediate representation after the attention module. This indicates a multi-head attention module. Feedforward network Representation layer normalization operation. First, input features... Entering the multi-head self-attention module This module learns the information relationships between different locations and compares this output with the original input. The sum is added together to form a residual connection. This helps alleviate the vanishing gradient problem in deep networks and also helps the model retain original information. Then, this summation result is normalized. This step stabilizes the training process and accelerates convergence; the output of this step is... This is the intermediate representation after this attention module. Next, Feed into a feedforward network In this module, the MLP performs a non-linear transformation on the feature vector at each location, enhancing the model's expressive power, and combines the MLP's output with its input. The results are added together to form a second residual connection; finally, the added result is normalized again to obtain the first residual connection. l Output feature data of layer encoder .

[0031] Specifically, such as Figure 1As shown, the spatiotemporal feature extraction module mainly includes a multi-scale convolution module, a channel attention module, and a spatial domain convolution module. The multi-scale convolution module uses several 1D convolution kernels of varying sizes to perform independent convolution operations on the input EEG signal. The convolution operations employ "same convolution." Furthermore, the feature maps obtained from each convolution operation are summed to obtain a multi-scale feature map (i.e., a temporal domain feature map) of the EEG signal. Then, the channel attention module assigns different channel weights to the feature channels in the feature map obtained from the temporal domain feature extraction, highlighting key features and ignoring redundancy. The channel attention module first uses Global Average Pooling (GAP) to compress the feature maps output by the multi-scale convolution module by channel, obtaining global features for each channel. Then, two fully connected layers of the same size are used to learn the relationships between channels. A non-linear activation function, Sigmoid, is used to obtain channel attention weights. These weights are multiplied by the corresponding channel features to output a weighted feature map, the size of which is the same as the unweighted map. The weighted feature map is then batch normalized and applied to the ReLU function before being fed into the spatial domain convolution module. The spatial domain convolution module also uses SAME convolution to obtain the spatial feature map of the EEG signal, and batch normalizes the obtained spatial feature map using Batch Normalization and then applies the ReLU function to obtain the spatial feature map of the EEG signal.

[0032] Specifically, the fusion process of frequency domain feature maps, temporal domain feature maps, and spatial domain feature maps is as follows: First, the feature maps of different domains are decoupled and unfolded along their sample dimensions, then concatenated along the channel dimension and aligned along the spatial dimension to obtain the concatenated multi-domain features; then, the concatenated multi-domain features are divided into sliding window blocks through an image patch embedding layer, and the feature vectors in each window are mapped to the latent space through a learnable linear projection to generate an embedding vector sequence with positional encoding information, and a class token is introduced as a trainable parameter and inserted into the beginning of the sequence; finally, multi-domain feature fusion is achieved through a stacked Transformer encoder.

[0033] The source domain dataset after preprocessing in step 1 is input into the multi-view feature extraction network for training, and a pre-trained network model can be obtained.

[0034] Specifically, during network training, in order to align the feature distributions corresponding to the feature maps in the frequency domain, time domain, and spatial domain, the following domain loss is introduced. : (17) in, This represents the signal samples in the source domain dataset. This represents the signal samples in the target domain dataset. This indicates the number of feature maps, here M =3, which represents the frequency domain feature map output by the multi-domain feature extraction module, the time domain feature map obtained by the spatiotemporal feature extractor module, and the spatial domain feature map. i, j This represents the sequence number of the signal sample in the source domain dataset. p , q This represents the sequence number of the signal sample in the target domain dataset. The multi-kernel function representing two signals is calculated using the following formula: (18) in, It's about signals. x and y Different kernel functions (such as Gaussian radial basis kernel function, etc.). It is the corresponding coefficient, and By weighted summing of the results from multiple different kernel functions, the characteristics of the data can be captured more comprehensively, thereby improving the accuracy of the estimation of distributional differences. k The number of kernel functions.

[0035] The classification loss is evaluated using the cross-entropy loss function, i.e.: (19) in, Represents classification loss. Indicates the first i One signal sample, Indicates the first i The label of each signal sample, The total number of samples, This represents a multi-view feature extraction network. This represents the function for calculating cross-entropy loss.

[0036] To fully utilize the features in the labels and domains, a composite loss function is used as the final loss function of the model: (20) in, This represents the total network loss.

[0037] During model training, the Adam optimization algorithm is used to minimize the composite loss function, enabling the model to learn feature representations with good generalization performance and better align the feature distributions of the source and target domains.

[0038] 3. Network fine-tuning A modular transfer adaptation strategy is used to adjust the multi-view feature extraction network trained in step 2. Specifically, for example... Figure 2As shown, this invention introduces the concept of transfer score vectors to quantitatively evaluate the modules that extract frequency domain feature maps, time domain feature maps, and spatial domain feature maps in a multi-view feature extraction network. A Gaussian mixture model is used to perform cluster analysis on the transfer score vectors. Based on the clustering results, it is determined whether the module parameters need fine-tuning. The clustering results are divided into two categories: modules with good transferability, represented by low-mean clusters, are designated as the frozen class; other transfer vectors are classified as the fine-tuning class. The target domain training dataset is input into the multi-view feature extraction network for retraining. During training, the parameters of the feature acquisition modules corresponding to the views of the frozen class transfer vectors remain unchanged, while the parameters of the feature acquisition modules corresponding to the views of the fine-tuning class transfer vectors are adjusted, thus completing the network fine-tuning.

[0039] Specifically, the migration score vector consists of the distribution difference scores between the source domain and the target domain. Discrimination score of target domain The composition, and its specific calculation process are as follows: Step (1): Calculate the following formula: m Feature map The signal sample distribution difference score between the source domain dataset and the target domain dataset is as follows. : (twenty one) in, This represents the signal samples in the source domain dataset. This represents the signal samples in the target domain dataset. i, j This represents the sequence number of the signal sample in the source domain dataset. p , q This represents the sequence number of the signal sample in the target domain dataset. m =1,2,…, M , (·,·) is a positive definite kernel function, and a Gaussian kernel function is generally used. , x and y This represents two signals in the input space. It is the bandwidth parameter, which controls the width of the function and determines the rate at which the function value decays. The Gaussian kernel function measures the similarity between two input signals in this way. Signals with high similarity will have higher values ​​under the Gaussian kernel function, while signals with low similarity will have lower values ​​under the Gaussian kernel function. Step (2): Calculate the following formulas respectively. m A view The average interclass distance between different class signal samples in the target domain dataset below and average intra-class distance : (twenty two) (twenty three) in, express distance, C This represents the total number of classes of signal samples in the target domain dataset. This represents calculating the central feature vector of all signal samples in a certain category. Indicate category All signal samples, Indicate category All signal samples, Indicate category c All signal samples, Indicates the category All samples Enter the number m After being processed by the view feature extractor, the resulting set of feature vectors for all samples of that category is obtained. Indicates the category All samples Enter the number m After being processed by the view feature extractor, the set of feature vectors for all samples of that class is obtained. Indicates the category c All samples Enter the number m After being processed by the view feature extractor, the resulting set of feature vectors for all samples of that category is obtained. c 1. c 2. c The table lists the category numbers respectively. Indicate category c The total number of signal samples contained in i Indicates the signal sample number. Indicate category c The i-th signal sample in the data. Indicates the category c The i signal samples Enter the number m After being processed by the view feature extractor, the feature vector set of all samples of that category is obtained; This represents calculating the central feature vector of all signal samples in a certain category. For category A... c All signal samples , its first m The formula for calculating the central eigenvector under each feature map is: (twenty four) Step (3): Calculate the following formula: m A view Discriminant scores of the target domain dataset below : (25) Step (4): with As the first m The transfer score vector of the feature extraction module corresponding to each view.

[0040] Specifically, the process of clustering the migration score vector using a Gaussian mixture model is as follows: the optimal mean is found by maximizing the log-likelihood function. Covariance Matrix This clusters the transfer score vectors into two categories: frozen and fine-tuned, as expressed by the formula: (26) in, It represents the set of migration score vectors across all views. , This represents the clustering result of all migration score vectors. If the output is 0, the input is considered to be in the frozen category; if the output is 1, the input is considered to be in the fine-tuning category. It is clustering c The mean vector, It is the corresponding covariance matrix. It is clustering c The probability, It is the first m The transfer score vector of the feature extraction module corresponding to each view; c =1 or 2 represent the freeze class and the fine-tuning class, respectively.

[0041] 4. Classification of electroencephalogram (EEG) signals The EEG signal dataset to be classified is input into the adjusted multi-view feature extraction network to obtain the classification results.

[0042] Based on the data from this embodiment, the effectiveness of the method of the present invention is verified in the following manner: Considering the high cost of EEG signal acquisition and the potential overfitting problem in deep learning models due to the scarcity of single-subject sample sizes, the EEG signal data was augmented using the EEGGMix method during both the training and testing phases. This means that for a given set of EEG signals of the same category... X Randomly select a pair of EEG signals and New EEG signals were obtained by mixing them using EEGGMix: (27) in, This indicates that mixing yields new EEG signals. Represents a pair of brain signals and Mixing ratio ,in , This indicates the group of EEG signals X Global standard deviation This indicates the group of EEG signals X The global average, Indicates random selection of EEG signals The average value, Indicates random selection of EEG signals The standard deviation.

[0043] The effectiveness of the method of the present invention is evaluated using the classification accuracy (Acuraccy), which is defined as follows: (28) Among them, TP (true positive) is the number of samples that are actually positive and correctly classified, TN (true negative) is the number of samples that are actually negative and correctly classified, FP (false positive) is the number of samples that are actually negative but are misclassified as positive, and FN (false negative) is the number of samples that are actually positive but are misclassified as negative.

[0044] The Kappa coefficient is used to assess classification consistency, taking into account accidental consistency. The calculation formula is as follows: (29) in, p o Indicates overall accuracy. p e This indicates the expected accuracy.

[0045] Table 1 shows the classification accuracy results of the method of this invention and other state-of-the-art (SOTA) methods on the BCI Competition IV2a dataset, and Table 2 shows the classification accuracy results on the BCI Competition IV 2b dataset. In the tables, DAN represents a Deep Adaptation Network (DAN) architecture proposed by Long et al., DANN represents a Domain-Adversarial Neural Network (DANN) proposed by Yaroslav et al., DRDA represents an end-to-end domain adaptation (DRDA) method based on deep representation proposed by Zhao et al., considering the differences in edge and conditional distributions between the source and target domains, DJDAN represents a Dynamic Joint Domain Adaptive Network (DJDAN) proposed by Hong et al., and CLUDA represents a deep domain adaptive framework based on CORAL loss (DDAF-CORAL) proposed by Zhong et al. to address the distribution differences in EEG signals caused by subject-related and time-related variations. Average / std represents the average / standard deviation of the classification accuracy of the corresponding method on different individuals. It can be seen that the present invention achieves higher classification accuracy compared to benchmark methods.

[0046] Table 1

[0047] Table 2

Claims

1. A modular adaptive EEG signal classification method based on multi-view features, characterized in that... The steps are as follows: Step 1, EEG signal preprocessing: For existing EEG signal data containing different subjects performing different imagination tasks, the EEG signals of the corresponding task categories are extracted and filtered according to the labels. The mean baseline correction method is used to correct the baseline of the filtered data. The ICA method is used to remove eye movement artifacts from the baseline corrected data, and then the data is normalized. The target domain dataset is composed of all EEG signals from a single subject, and the source domain dataset is composed of all EEG signals from other subjects. The target domain dataset is divided into a training set and a validation set according to a 4:1 ratio of the number of EEG signal samples. Step 2, Network Training: The multi-view feature extraction network is trained using the source domain dataset after preprocessing in Step 1 as input and the EEG signal classification results as output. The multi-view feature extraction network includes a frequency domain feature extraction module and a spatiotemporal feature extraction module deployed in parallel, as well as a classifier module. The frequency domain feature extraction module extracts the frequency domain feature map of the input data, and the spatiotemporal feature extraction module extracts the temporal domain feature map and the spatial domain feature map of the input data. The feature maps from different domains are fused and input into the classifier module. The features are mapped to the target class space through a fully connected classification head, and the probability distribution is calculated using the Softmax function to obtain the classification result. The specific processing procedure of the frequency domain feature extraction module is as follows: First, input the electroencephalogram (EEG) signal. ,right X Perform frequency band filtering to obtain filtered data. ,in, C Indicates the number of channels. T Indicates the number of sampling points. N b This indicates the number of frequency band filters; then, the differential entropy features of the filtered data are extracted. and power spectral density characteristics Then, the features are spliced ​​together to obtain the spliced ​​features. ,in, N s Indicates the number of segments in the time dimension. N psd This represents the feature length of the single-band power spectral density (PSD); then, the stitched feature F is input into the linear projection layer, according to... Calculate the feature data ,in, , D This represents the feature dimension generated after linear projection; finally, the feature data... z lp After adding position encoding, the input is a feature fusion network consisting of several Transformer encoders connected in series, resulting in the final frequency domain feature map; The spatiotemporal feature extraction module includes a multi-scale convolution module, a channel attention module, and a spatial convolution module. The multi-scale convolution module uses several 1D convolution kernels of varying sizes to perform independent convolution operations on the input EEG signal. The feature maps obtained from each convolution operation are summed to obtain the temporal feature map of the EEG signal. The temporal feature map is then processed by the channel attention module to obtain a weighted feature map. This weighted feature map undergoes batch normalization and a ReLU activation function before being fed into the spatial convolution module. The spatial convolution module uses a 3×1×1 convolution kernel to perform convolution operations on the input feature map, followed by batch normalization and a ReLU activation function to obtain the spatial feature map of the EEG signal. The specific implementation process of fusing features from different domains in the multi-view feature extraction network is as follows: First, the feature maps of different domains are decoupled and unfolded along their sample dimensions, then concatenated along the channel dimension and aligned along the spatial dimension to obtain the concatenated multi-domain features; then, the concatenated multi-domain features are divided into sliding window blocks through an image patch embedding layer, and the feature vectors in each window are mapped to the latent space through a learnable linear projection to generate an embedding vector sequence with positional encoding information, and a class identifier is introduced as a trainable parameter and inserted into the beginning of the sequence; finally, multi-domain feature fusion is achieved through a stacked Transformer encoder. Step 3, Network Fine-tuning: The multi-view feature extraction network trained in Step 2 is adjusted using a modular transfer adaptation strategy; Step 4, EEG signal classification: Input the EEG signal dataset to be classified into the adjusted multi-view feature extraction network to obtain the classification results.

2. The modular adaptive EEG signal classification method based on multi-view features as described in claim 1, characterized in that: The filtering process includes first using a Butterworth bandpass filter from 0.5Hz to 100Hz to filter out high-frequency noise interference in the EEG signal, and then using a 50Hz notch filter to suppress power frequency noise in the EEG signal; the data normalization process uses z-score normalization.

3. The modular adaptive EEG signal classification method based on multi-view features as described in claim 1, characterized in that: The differential entropy feature extraction method is as follows: for continuous random variables X If its probability density function is f ( x Its differential entropy characteristics are: ; The power spectral density feature extraction method is as follows: First, the original signal... x ( n Divide into uniform segments, with the number of segments being... L Each segment is [length] M Overlapping sections are set between segments, with the number of overlap points... Pick M / 2, apply a window function to each signal segment. The window function value smoothly decays to zero at the segment endpoints; then, a discrete Fourier transform is performed on each windowed signal segment, and the power spectral density estimate is obtained according to Parseval's theorem. （1） in, Indicates the first m segment signal x m ( n At frequency point k The power spectral density estimate at that location, , The complex exponential basis used in calculating the Fourier transform, U The normalization factor is calculated using the following formula: ; Finally, according to Calculated signal x ( n The final power spectral density estimate of ).

4. The modular adaptive EEG signal classification method based on multi-view features as described in claim 1, characterized in that: The convolution operation described uses same convolution; The channel attention module first uses global average pooling to compress the feature map output by the multi-scale convolution module by channel to obtain the global features of each channel. Then, it uses two fully connected layers of the same size to learn the relationship between channels, uses the non-linear activation function Sigmoid to obtain the channel attention weights, and multiplies the channel attention weights with the features of the corresponding channels to output the weighted feature map.

5. The modular adaptive EEG signal classification method based on multi-view features as described in claim 1, characterized in that: The network training uses the following total loss function: （2） in, Indicates the total network loss. Indicates domain loss, Indicates classification loss; The aforementioned domain loss The calculation formula is as follows: （3） in, This represents the signal samples in the source domain dataset. This represents the signal samples in the target domain dataset. Indicates the number of feature maps. M =3, representing the frequency domain feature map, time domain feature map, and spatial domain feature map, respectively. i, j This represents the sequence number of the signal sample in the source domain dataset. p , q This represents the sequence number of the signal sample in the target domain dataset. The multi-kernel function representing two signals is calculated using the following formula: （4） in, It is the first u A signal x and y kernel function, The corresponding coefficients satisfy... , k The number of kernel functions; The classification loss The calculation formula is as follows: （5） in, Indicates the first i One signal sample, Indicates the first i The label of each signal sample The total number of samples, This indicates the multi-view feature extraction network. This represents the calculation of the cross-entropy loss function; During training, the best network model is obtained by minimizing the total loss function using the Adam optimization algorithm.

6. The modular adaptive EEG signal classification method based on multi-view features as described in claim 1, characterized in that: The specific process of adjusting the network using the modular transfer adaptation strategy is as follows: First, calculate the transfer score vectors of the feature extraction modules corresponding to different views. Then, using the transfer score vectors as samples, all transfer score vectors form a new sample set. Use a Gaussian mixture model to perform cluster analysis on the samples in this set, dividing them into two classes: the low-mean clustered samples are designated as the frozen class, and the other samples are designated as the fine-tuning class. Finally, input the target domain training dataset into the multi-view feature extraction network for retraining. During training, keep the parameters of the feature extraction modules for the views corresponding to the frozen class samples unchanged, and adjust the parameters of the feature acquisition modules for the views corresponding to the fine-tuning class samples to complete the network fine-tuning.

7. A modular adaptive EEG signal classification method based on multi-view features as described in claim 1 or 3, characterized in that: The Transformer encoder is a composite computational unit that includes a multi-head attention mechanism, layer normalization, residual connections, and a feedforward network. It establishes long-range dependencies between cross-domain features through a multi-head self-attention mechanism, quantifies the correlation strength between frequency domain features and time domain features using a cross-attention weight matrix, and achieves the fusion of multi-domain features through cascaded multi-layer nonlinear transformations at each encoder level.

8. The modular adaptive EEG signal classification method based on multi-view features as described in claim 6, characterized in that: The migration score vector is composed of the distribution difference scores between the source and target domains. Discrimination score of target domain The composition, and its specific calculation process are as follows: Step (1): Calculate the following formula: m Feature map The signal sample distribution difference score between the source domain dataset and the target domain dataset is as follows. : （6） in, This represents the signal samples in the source domain dataset. This represents the signal samples in the target domain dataset. i, j This represents the sequence number of the signal sample in the source domain dataset. p , q This represents the sequence number of the signal sample in the target domain dataset. m =1,2,…, M , (·,·) is a positive definite kernel function, using a Gaussian kernel function. , x and y This represents two signals in the input space. It is a bandwidth parameter; Step (2): Calculate the following formulas respectively. m Feature map The average inter-class distance between different class signal samples in the target domain dataset below and average intra-class distance : （7） （8） in, express distance, C This represents the total number of classes of signal samples in the target domain dataset. This represents calculating the central feature vector of all signal samples in a certain category. Indicate category All signal samples, Indicate category All signal samples, Indicate category c All signal samples, Indicates the category All samples Input to extract the first m The feature vector set of all samples of this category obtained after the feature extraction module of each feature map. Indicates the category All samples Input to extract the first m The feature vector set of all samples of this category obtained after the feature extraction module of each feature map. Indicates the category c All samples Input to extract the first m The feature vector set of all samples of this category obtained after the feature extraction module of each feature map. c 1. c 2. c The table lists the category numbers respectively. Indicate category c The total number of signal samples contained in i Indicates the signal sample number. Indicate category c The first in i One signal sample, Indicates the category c The i signal samples Input to extract the first m The feature extraction module of each feature map obtains the features of the sample; This represents calculating the central feature vector of all signal samples in a certain category. For category A... c All signal samples , its first m The formula for calculating the central eigenvector under each feature map is: （9） Step (3): Calculate the following formula: m A view Discriminant scores of the target domain dataset below : （10） Step (4): with As the first m The transfer score vector of the feature extraction module corresponding to each view.

9. The modular adaptive EEG signal classification method based on multi-view features as described in claim 6, characterized in that: The process of clustering migration score vectors using a Gaussian mixture model is as follows: the optimal mean is found by maximizing the log-likelihood function. Sum of covariance matrix This clusters the transfer score vectors into two categories: frozen and fine-tuned, as expressed by the formula: （11） in, It represents the set of migration score vectors across all views. , This represents the clustering result of all migration score vectors. If the output is 0, the input is considered to be in the frozen category; if the output is 1, the input is considered to be in the fine-tuning category. It is clustering c The mean vector, It is the corresponding covariance matrix. It is clustering c The probability, It is the first m The transfer score vector of the feature extraction module corresponding to each view; c =1 or 2 represent the freeze class and the fine-tuning class, respectively.

Images