Cross-temporal brain pattern recognition method based on tensorized spatial-frequency attention domain adaptation network

Abstract

The present invention discloses a cross-time brainprint recognition method based on a tensorized frequency-space attention domain adaptation network. Most existing multi-source domain adaptation methods are to bridge the domain gap between multiple source domains and target domains individually, while ignoring the relationship between the invariant features of each distribution alignment domain. The present invention assists the performance of the target domain by modeling the important relationship of domain-invariant features without being affected by the distribution differences between source domains. A new tensorized frequency-space attention network (TSFAN) is adopted to jointly merge paired sources and targets and appropriate common frequency-space features across source domains. Taking into account the dimensionality problem, TSFAN is further approximately represented as a low-rank Tucker format, so that TSFAN can be linearly expanded in the number of domains, and TSFAN can be extended to the case of any number of time periods. The present invention can achieve efficient cross-time task-independent brainprint recognition, and is an effective method for portable brainprint recognition that can be used in real life.

Description

Technical Field

[0001] The present invention belongs to the field of EEG signal recognition in the field of biometric recognition, and specifically relates to a cross-time period brain pattern recognition method based on a tensorized spectral-spatial attention domain adaptation network. Specifically, a multi-source domain adaptation network based on tensorized spectral-spatial attention is introduced to extract paired transferable features between source domains and target domains and interactive information between multiple domains, thereby mining stable and reliable EEG identity features for unsupervised classification. Background Art

[0002] Biometrics rely on personal characteristics and play a key role in identity authentication systems. Although physical biometrics, such as facial recognition and fingerprint recognition, have been widely used in real life, the potential danger of careful forgery or secret copying is still unavoidable. In addition to physical biometrics, brain activity recorded by electroencephalogram (EEG) signals has been proposed as a new cognitive biometric that meets basic identity recognition requirements. In addition, only living individuals can provide signals of brain activity, and these signals are not under the control of the user. This means that the user's identity information cannot be deliberately leaked or stolen, making EEG-based biometrics suitable for applications with high security requirements.

[0003] Reliable and stable EEG identity features are the basis of EEG-based biometric recognition. In fact, in a large number of studies, traditional machine learning methods are used, which seriously require expertise to extract features and are always insufficient for good performance. In recent years, deep learning has attracted considerable attention in decoding EEG identification features due to its ability to capture high-level features and potential dependencies. In general, various types of deep learning methods such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and graph convolutional neural networks (GCNNs) have been shown to obtain temporal, frequency, and spatial identity discrimination features from EEG signals.

[0004] EEG signals are unstable across sessions due to factors such as impedance, small displacements of electrode positions, and changes in subject states. Therefore, despite these significant advances, cross-session based biometrics in real-world scenarios remains challenging. Most previous studies have focused on session or mixed multi-session data, ignoring the distribution differences between EEG data from multiple training sessions. Intuitively, it is not easy to eliminate the offset of domain-invariant representation extraction even between a single source domain (training session) and target domain (testing session) data, and a greater degree of mismatch across multiple source domains may lead to unsatisfactory performance.

[0005] To avoid the influence of domain shift between multiple source domains, EEG signal multi-source domain adaptation methods minimize the differences between source and target domains respectively. In fact, domain-invariant features captured with different source domains represent stable information from multiple views and transfer more appropriate information to the target domain. However, the domain-invariant features calculated by each distribution alignment are affected by the involved source domains and cannot benefit from the common relationship of multiple source domains.

[0006] To address these issues, the present invention proposes a cross-period brainprint recognition method (TSFAN) based on tensorized spatial-frequency attention domain adaptation network to capture stable EEG identity features across time periods. Specifically, each pair of source and target domain data is mapped to different temporal feature spaces. Then, the core idea of ​​TSFAN is designed - tensor-based attention is used to tensorize the spatial-frequency attention of the source and target domains to obtain domain-invariant spatial-frequency features, which naturally contribute to transferable information within the source and complex interactions between sources. Considering the curse of dimensionality, low-rank Tucker format tensors are further adopted to enable TSFAN to scale linearly in the number of domains. Summary of the invention

[0007] The purpose of this invention is to address the deficiencies of the prior art and propose a cross-time brainprint recognition method based on a tensorized spatio-spectral attention domain adaptation network. This method mainly constructs a tensorized spatio-spectral attention network based on multi-source domain adaptation, which fully utilizes the interactive correlation between different domains while alleviating the difference in data distribution between the source domain and the target domain in pairs.

[0008] A cross-time brain pattern recognition method based on a tensorized spatial-frequency attention domain adaptation network comprises the following steps:

[0009] Step (1), preprocessing raw EEG data;

[0010] 1-1 Under the same experimental paradigm, EEG data of multiple subjects under external stimulation at different time periods were collected;

[0011] 1-2 To remove the interference of external devices and electromyography, the original EEG data is filtered using a Butterworth filter and then subjected to a fast Fourier transform (STFT);

[0012] 1-3 intercepts the EEG data processed in step 1-2, and labels the corresponding EEG sample data with the labels of the subjects to which they belong;

[0013] 1-4 Divide the EEG sample data obtained after processing in step 1-3 into a training set and a test set in proportion, wherein the training set data contains K time period data, that is, K source domains, K ≥ 2; the test set is used as the target domain;

[0014] Step (2), constructing a tensor-based spatial-frequency attention domain adaptation network model, and training and testing it;

[0015] The domain adaptation network model based on tensorized frequency-space attention includes K specific domain feature extraction networks with the same structure and 1 tensorized frequency-space attention network. Each specific domain feature extraction network includes a multi-scale one-dimensional convolution layer, a splicing layer, a maximum pooling layer, a fusion layer, and a frequency-space convolution layer. The multi-scale one-dimensional convolution layer includes multiple parallel one-dimensional convolutions of different scales. The frequency-space convolution layer includes a frequency domain one-dimensional convolution and a space domain one-dimensional convolution connected in series.

[0016] The input of the multi-scale one-dimensional convolution layer is a source domain data and a target domain data, and its output is to the concatenation layer;

[0017] The splicing layer splices the received multiple features of different scales to obtain the source domain brain pattern time domain feature Zt sj and the target domain brain pattern time domain feature Zt tj , j∈[1,K], and then output the above features to the maximum pooling layer and fusion layer respectively;

[0018] The maximum pooling layer performs dimensionality reduction processing on the received features in the time dimension, and then outputs them to the tensorized spatial-frequency attention network;

[0019] The tensorized spatial-frequency attention network receives the features output by the maximum pooling layer of K specific domain feature extraction networks, and interactively processes the above features to obtain the source domain spatial-frequency attention Q containing the interactive correlation between features. sj and the target domain spatial-frequency attention Q tj , and then output the above attention to the fusion layer; specifically:

[0020] The tensorized spatial-frequency attention network uses two fully connected layers to implement nonlinear mapping of the features output by the K specific domain feature extraction networks to obtain the source domain spatial-frequency attention Q sj and the target domain spatial-frequency attention Q tj :

[0021] Q sj =F bj (Relu(F aj (P sj ; V j ));u j ) Formula (1)

[0022] Q tj =F bj (Relu(F aj (P tj ; V j ));u j )

[0023] Among them, F aj and F bj represents the two fully connected layers in the j-th source domain space, V j and U j Represents the parameters of the two fully connected layers, Relu(.) is the activation function, Represents the spatial frequency characteristics of the source domain and target domain output by the maximum pooling layer; c is the number of original feature EEG channels, and s is the size of the original feature frequency domain dimension;

[0024] The parameters of the fully connected layer in formula (1) The tensor is quantized to a (K+1)-order high-order tensor , c' is the fully connected layer F aj The number of EEG channels of the processed features, s' is the fully connected layer F aj The frequency domain dimension size of the processed features is used to obtain the interactive correlation between features. Considering that the increase in the number of source domains may cause dimensional disasters, the low-rank Tucker form is used to represent high-order tensors.

[0025]

[0026] in {r 1 ...r K+1} is the rank of the Tucker form, I 1 =I 2 =...=I K =c′s′,I K+1 =cs; c is the number of original feature EEG channels, s is the size of the original feature frequency domain dimension;

[0027] The fusion layer receives the source domain brain pattern time domain feature Zt sj and the target domain brain pattern time domain feature Zt tj Respectively with the source domain frequency-space attention Q sj and the target domain spatial-frequency attention Q tj Fusion is performed to obtain the frequency-space enhanced source domain brain pattern time domain feature Zt' sj and the target domain brain pattern time domain feature Zt′ tj , and output to the frequency-space convolution layer;

[0028] The frequency-space convolution layer receives the time domain feature Zt' sj , Zt' tj Through one-dimensional convolution in frequency domain and one-dimensional convolution in spatial domain, the source domain time-frequency spatial brain pattern feature Z is extracted. sj and the target domain time-frequency empty brain pattern feature Z tj ;

[0029] Step (3), constructing a classifier for brain pattern recognition, and training and testing it;

[0030] The time-frequency-spatial feature Z output from step 2 sj ,Z tj Flatten, pass the fully connected layer and Softmax activation function to calculate the probability that the sample belongs to each category;

[0031] Step (4), using the trained and tested tensor-based spatial-frequency attention domain adaptation network model and the classifier for brain pattern recognition to achieve cross-time brain pattern recognition.

[0032] Preferably, step 1-2 uses a Butterworth filter to filter the original EEG data, specifically downsampling the EEG data to 250 Hz, and using a Butterworth filter to filter the original EEG data at 0 to 75 Hz.

[0033] Preferably, the fast Fourier transform in step 1-2 is to perform short-time Fourier transform on the filtered signal x to extract time-frequency features:

[0034] Using a time-limited window function h(t), assuming that the non-stationary signal x is stationary within a time window, the signal x is analyzed segment by segment through the movement of the window function h(t) on the time axis to obtain a set of local "spectra" of the signal; the short-time Fourier transform of the signal x(τ) is defined as:

[0035]

[0036] Where STFT(t, f) represents the short-time Fourier transform of the signal x(τ) at time t, h(τ-t) is the window function, and f represents the frequency.

[0037] Preferably, the fusion layer is:

[0038]

[0039]

[0040] Preferably, the loss function of the classifier for brain pattern recognition is for:

[0041]

[0042] where θ y is the classifier parameter, N is the number of categories, θ f represents the feature extractor parameters, represents the cross entropy loss of the i-th class, and E() represents the cross entropy function.

[0043] As a preferred method, the total loss function of the classifier pair based on the tensorized spatial-frequency attention domain adaptation network model and brain pattern recognition is for:

[0044]

[0045] in Represents the distance loss function used to measure the classifier. It represents the loss function used to measure the difference in data distribution between the source domain and the target domain, and λ and γ are hyperparameters.

[0046] Another object of the present invention is to provide a cross-period brainprint recognition device, comprising:

[0047] The EEG data preprocessing module is used to filter and perform fast Fourier transform on the collected EEG data at different time periods;

[0048] The trained and tested tensor-based frequency-space attention domain adaptation network model is used to extract features from the EEG data of different time periods output by the EEG data preprocessing module to obtain the source domain time-frequency-space brain pattern features Z sj and the target domain time-frequency empty brain pattern feature Z tj ;

[0049] Train and test the classifier for brain pattern recognition, and use the source domain time-frequency spatial brain pattern feature Z sj and the target domain time-frequency empty brain pattern feature Z tj Flatten, through the fully connected layer and Softmax activation function, calculate the probability that the sample belongs to each category to achieve cross-time brain pattern recognition.

[0050] Another object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the above method.

[0051] Another object of the present invention is to provide a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the above method is implemented.

[0052] The beneficial effects of the present invention are:

[0053] The present invention proposes to jointly capture the intra-source transferable information and cross-source interaction of domain-invariant features to alleviate the degradation of discrimination caused by global distribution alignment, and proposes a tensor-based attention mechanism to tensorize the attention of a specific field in a low-rank Tucker format, so that it can interact between multi-source views without being affected by the curse of dimensionality. The method of the present invention is expected to be applied as brainprint recognition in biometrics with a high degree of confidentiality. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flow chart of the brain pattern recognition model proposed by the present invention;

[0055] Figure 2 This is the architecture diagram of the tensorized spatial-frequency attention domain adaptation network proposed in the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention more clear, the following is a further detailed description in combination with the technical solution and drawings of the present invention:

[0057] The present invention relates to a method for cross-time brain pattern recognition based on a tensorized frequency-space attention domain adaptation network, and its flow chart is as follows: Figure 1 The model architecture diagram is shown in Figure 2 As shown in the figure, it consists of two modules: (1) intra-source transferable feature learning, which uses a temporal feature extractor and a spatial frequency feature extractor for multi-source domain adaptation to obtain domain-invariant features for each pair of source and target domains; (2) tensorized spatial-frequency attention, which is used to model complex inter-source interactions. The entire architecture is carefully designed to explore stable EEG recognition features across sessions.

[0058] Step 1: Preprocessing raw EEG data

[0059] 1) The noise frequency contained in the original EEG signal is usually lower than 0.5Hz or higher than 50Hz. In order to remove the power frequency interference caused by the EEG acquisition equipment and the EMG interference of the subjects, the EEG data is downsampled to 250Hz and the original EEG data is filtered from 0 to 75Hz using a Butterworth filter;

[0060] 2) Perform a short-time Fourier transform on the signal x output from operation 1) to extract the time-frequency features. Using a time-limited window function h(t), assuming that the non-stationary signal x is stationary within a time window, the signal x is analyzed segment by segment by moving the window function h(t) on the time axis to obtain a set of local "spectra" of the signal. The specific window size of this scheme is 0.5s. The short-time Fourier transform of the signal x(τ) is defined as:

[0061]

[0062] Where STFT(t, f) represents the short-time Fourier transform of the signal x(τ) at time t, where h(τ-t) is the window function and f represents the frequency.

[0063] 3) Using a 15-s time window to intercept the EEG data processed in 2), and label the corresponding EEG sample data with the subject's label;

[0064] 4) Divide the EEG sample data obtained after processing in 3) into training sets in proportion and the test set {X t , Y t}, where K is the number of time periods. EEG samples Among them, c is the number of EEG channels, s is the size of the frequency domain dimension, and t is the size of the time domain dimension. Specifically, this scheme selects nine channels: Fz, F7, F8, C3, C4, P7, P8, O1, and O2, with a sampling rate of 1-30Hz and a sampling rate of 250Hz, that is, c=9, s=30, and t=30.

[0065] Step 2: Construct a tensor-based frequency-space attention domain adaptation network model;

[0066] The domain adaptation network model based on tensorized frequency-space attention includes K specific domain feature extraction networks with the same structure and 1 tensorized frequency-space attention network. Each specific domain feature extraction network includes a multi-scale one-dimensional convolution layer, a splicing layer, a maximum pooling layer, a fusion layer, and a frequency-space convolution layer. The multi-scale one-dimensional convolution layer includes multiple parallel one-dimensional convolutions of different scales. The frequency-space convolution layer includes a frequency domain one-dimensional convolution and a space domain one-dimensional convolution connected in series.

[0067] The input of the multi-scale one-dimensional convolution layer is a source domain data and a target domain data, and its output is to the concatenation layer;

[0068] The splicing layer splices the received multiple features of different scales to obtain the source domain brain pattern time domain feature Zt sj and the target domain brain pattern time domain feature Zt tj , j∈[1,K], and then output the above features to the maximum pooling layer and the fusion layer respectively;

[0069] The maximum pooling layer performs dimensionality reduction processing on the received features in the time dimension, and then outputs them to the tensorized spatial-frequency attention network;

[0070] The tensorized spatial-frequency attention network receives the features output by the maximum pooling layer of K specific domain feature extraction networks, and interactively processes the above features to obtain the source domain spatial-frequency attention Q containing the interactive correlation between features. sj and the target domain spatial-frequency attention Q tj , and then output the above attention to the fusion layer; specifically:

[0071] The tensorized spatial-frequency attention network uses two fully connected layers to implement nonlinear mapping of the features output by the K specific domain feature extraction networks to obtain the source domain spatial-frequency attention Q sj and the target domain spatial-frequency attention Q tj :

[0072] Q sj =F bj (Relu(F aj (Psj ; V j ));u j ) Formula (1)

[0073] Q tj =F bj (Relu(F aj (P tj ; V j ));u j )

[0074] Among them, F aj and F bj represents the two fully connected layers in the j-th source domain space, V j and U j Represents the parameters of the two fully connected layers, Relu(.) is the activation function, Represents the spatial frequency characteristics of the source domain and target domain output by the maximum pooling layer; c is the number of original feature EEG channels, and s is the size of the original feature frequency domain dimension;

[0075] The parameters of the fully connected layer in formula (1) The tensor is quantized to a (K+1)-order high-order tensor , c' is the fully connected layer F aj The number of EEG channels of the processed features, s' is the fully connected layer F aj The frequency domain dimension size of the processed features is used to obtain the interactive correlation between features. Considering that the increase in the number of source domains may cause dimensional disasters, the low-rank Tucker form is used to represent high-order tensors.

[0076]

[0077] in {r 1 ...r K+1} is the rank of the Tucker form, I 1 =I 2 =...=I K =c′s′,I K+1 =cs; c is the number of original feature EEG channels, s is the size of the original feature frequency domain dimension;

[0078] The fusion layer receives the source domain brain pattern time domain feature Zt sj and the target domain brain pattern time domain feature Zt tj Respectively with the source domain frequency-space attention Q sj and the target domain spatial-frequency attention Q tj Fusion is performed to obtain the frequency-space enhanced source domain brain pattern time domain feature Zt' sj and the target domain brain pattern time domain feature Zt'tj , and output to the frequency-space convolution layer; specifically:

[0079]

[0080]

[0081] The frequency-space convolution layer receives the time domain feature Zt' sj , Zt' tj Through one-dimensional convolution in frequency domain and one-dimensional convolution in spatial domain, the source domain time-frequency spatial brain pattern feature Z is extracted. sj and the target domain time-frequency empty brain pattern feature Z tj ;

[0082] Step 3: Construct a classifier for brain pattern recognition;

[0083] Flatten the time-frequency-spatial features output from step 2, and calculate the probability that the sample belongs to each category through the fully connected layer and the Softmax activation function. The loss function of the classifier is defined as

[0084]

[0085] where θ y is the classifier parameter, and N is the number of categories.

[0086] Step 4: Train the network model

[0087] Using the training set obtained in step 1.4, perform gradient back propagation to optimize the loss function of the model constructed in steps 2 to 3, and save the best model through the validation set obtained in step 1.4 for testing. The loss function is expressed as:

[0088]

[0089] in Used to measure the classifier distance, It is used to measure the difference in data distribution between the source domain and the target domain. λ and γ are hyperparameters, which are set to 0.5 in the present invention. The SGD optimizer is used, the learning rate is 0.025, and the batch_size is 64.

[0090] Step 7: Verify the effectiveness of this scheme on a multi-task identity recognition dataset, which includes 30 subjects N=30. The scheme is verified by leaving the first period and the last period data as test data. The scheme is compared with the existing domain merging and multi-source domain methods. The results are shown in Table 1. The verification results show that the model proposed in this invention can effectively extract stable brain pattern features in different periods.

[0091] Table 1 Accuracy and error rate of the model on the cross-period identity recognition dataset

[0092]

[0093]

Claims

1. A cross-time brain pattern recognition method based on tensorized spatial-frequency attention domain adaptation network. Features The following steps are involved: Step (1), preprocessing raw EEG data; 1-1 Under the same experimental paradigm, EEG data of multiple subjects under external stimulation at different time periods were collected; 1-2 Use Butterworth filter to filter the raw EEG data, and then perform fast Fourier transform; 1-3 intercepts the EEG data processed in step 1-2, and labels the corresponding EEG sample data with the labels of the subjects to which they belong; 1-4 Divide the EEG sample data obtained after processing in step 1-3 into a training set and a test set in proportion, wherein the training set data contains K time period data, that is, K source domains, K ≥ 2; the test set is used as the target domain; Step (2), constructing a tensor-based spatial-frequency attention domain adaptation network model, and training and testing it; The domain adaptation network model based on tensorized frequency-space attention includes K specific domain feature extraction networks with the same structure and 1 tensorized frequency-space attention network. Each specific domain feature extraction network includes a multi-scale one-dimensional convolution layer, a splicing layer, a maximum pooling layer, a fusion layer, and a frequency-space convolution layer. The multi-scale one-dimensional convolution layer includes multiple parallel one-dimensional convolutions of different scales. The frequency-space convolution layer includes a frequency domain one-dimensional convolution and a space domain one-dimensional convolution connected in series. The input of the multi-scale one-dimensional convolution layer is a source domain data and a target domain data, and its output is to the concatenation layer; The splicing layer splices the received multiple features of different scales to obtain the source domain brain pattern time domain feature Zt sj and the target domain brain pattern time domain feature Zt tj , j∈[1,K], and then output the above features to the maximum pooling layer and fusion layer respectively; The maximum pooling layer performs dimensionality reduction processing on the received features in the time dimension, and then outputs them to the tensorized spatial-frequency attention network; The tensorized spatial-frequency attention network receives the features output by the maximum pooling layer of K specific domain feature extraction networks, and interactively processes the above features to obtain the source domain spatial-frequency attention Q containing the interactive correlation between features. sj and the target domain spatial-frequency attention Q tj , and then output the above attention to the fusion layer; specifically: The tensorized spatial-frequency attention network uses two fully connected layers to implement nonlinear mapping of the features output by the K specific domain feature extraction networks to obtain the source domain spatial-frequency attention Q sj and the target domain spatial-frequency attention Q tj : Q sj =F bj (Relu(F aj (P sj ; V j ));U j ) Formula (1) Q tj =F bj (Relu(F aj (P tj !V j ));HER j ) Among them, F aj and F bj represents the two fully connected layers in the j-th source domain space, V j and U j Represents the parameters of the two fully connected layers, Relu(.) is the activation function, Represents the spatial frequency characteristics of the source domain and target domain output by the maximum pooling layer; c is the number of original feature EEG channels, and s is the size of the original feature frequency domain dimension; The fully connected layer parameters in formula (1) The tensor is quantized to a (K+1)-order high-order tensor c' is the fully connected layer F aj The number of EEG channels of the processed features, s' is the fully connected layer F aj The frequency domain dimension size of the processed features is used to obtain the interactive correlation between features. Considering that the increase in the number of source domains may cause dimensional disasters, the low-rank Tucker form is used to represent high-order tensors. in {r 1 …r K+1 } is the rank of the Tucker form, I 1 =I 2 =…=I K =c's',I K+1 =cs; c is the number of original feature EEG channels, s is the size of the original feature frequency domain dimension; The fusion layer receives the source domain brain pattern time domain feature Zt sj and the target domain brain pattern time domain feature Zt tj Respectively with the source domain frequency-space attention Q sj and the target domain spatial-frequency attention Q tj Fusion is performed to obtain the frequency-space enhanced source domain brain pattern time domain feature Zt' sj and the target domain brain pattern time domain feature Zt' tj , and output to the frequency-space convolution layer; The frequency-space convolution layer receives the time domain feature Zt' sj , Zt' tj Through one-dimensional convolution in frequency domain and one-dimensional convolution in spatial domain, the source domain time-frequency spatial brain pattern feature Z is extracted. sj and the target domain time-frequency empty brain pattern feature Z tj ; Step (3), constructing a classifier for brain pattern recognition, and training and testing it; The source domain time-frequency empty brain pattern feature Z output in step 2 sj and the target domain time-frequency empty brain pattern feature Z tj Flatten, pass the fully connected layer and Softmax activation function to calculate the probability that the sample belongs to each category; Step (4), using the trained and tested tensor-based spatial-frequency attention domain adaptation network model and the classifier for brain pattern recognition to achieve cross-time brain pattern recognition.

2. The method according to claim 1, Features Step 1-2 uses a Butterworth filter to filter the raw EEG data. Specifically, the EEG data is downsampled to 250 Hz, and the raw EEG data is filtered at 0-75 Hz using a Butterworth filter.

3. The method according to claim 1, Features The fast Fourier transform in step 1-2 is specifically to perform a short-time Fourier transform on the filtered signal x to extract the time-frequency features: Using a time-limited window function h(t), assuming that the non-stationary signal x is stationary within a time window, the signal x is analyzed segment by segment through the movement of the window function h(t) on the time axis to obtain a set of local "spectra" of the signal; the short-time Fourier transform of the signal x(τ) is defined as: Where STFT(t,f) represents the short-time Fourier transform of the signal x(τ) at time t, h(τ-t) is the window function, and f represents the frequency.

4. The method according to claim 1, Features The fusion layer is specifically: 。 5. The method according to claim 1, Features The loss function of the classifier used for brain pattern recognition for: where θ y is the classifier parameter, N is the number of categories, θ f represents the feature extractor parameters, represents the cross entropy loss of the i-th class, and E() represents the cross entropy function.

6. The method according to claim 5, Features Total loss function of tensor-based spatial-frequency attention domain adaptation network model and classifier pair for brain pattern recognition for: in Represents the distance loss function used to measure the classifier. It represents the loss function used to measure the difference in data distribution between the source domain and the target domain, and λ and γ are hyperparameters.

7. A cross-time brain pattern recognition device implementing the method described in any one of claims 1 to 6, Features include: The EEG data preprocessing module is used to filter and perform fast Fourier transform on the collected EEG data at different time periods; The trained and tested tensor-based spatial-frequency attention domain adaptation network model is used to extract features from the EEG data of different time periods output by the EEG data preprocessing module to obtain the source domain spatial-frequency brain pattern features Z sj and the target domain time-frequency empty brain pattern feature Z tj ; Train and test the classifier for brain pattern recognition, and use the source domain time-frequency spatial brain pattern feature Z sj and the target domain time-frequency empty brain pattern feature Z tj Flatten, through the fully connected layer and Softmax activation function, calculate the probability that the sample belongs to each category to achieve cross-time brain pattern recognition.

8. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 6.

9. A computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 6 is implemented.

Images