Low-duration calibration electroencephalogram decoding method and system based on invariant feature extraction

Abstract

The invention discloses a low-duration calibration electroencephalogram decoding method and system based on invariant feature extraction, and relates to the technical field of electroencephalogram signal decoding, and the method comprises the steps: obtaining electroencephalogram data with labels, and carrying out Fourier transform to obtain amplitude information and phase information; inputting the phase information and the tag into a teacher network for training to obtain feature output of the teacher network; performing random pairing and reconstruction on the labeled electroencephalogram data to obtain reconstructed electroencephalogram data, inputting the reconstructed electroencephalogram data into a student network, and learning phase information extracted from a teacher network through the student network to obtain feature output of the student network; and acquiring real-time data, and inputting the real-time data into a student network to obtain an electroencephalogram decoding recognition result. Similarity features among different individuals are extracted through an invariant feature extraction method, when electroencephalogram signals of a new user are analyzed, the similarity features can be directly used for electroencephalogram analysis of a new subject, and the use efficiency of a non-intrusive brain-computer interface and the electroencephalogram recognition accuracy are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal decoding technology, and more specifically to a low-duration calibration EEG decoding method and system based on invariant feature extraction. Background Technology

[0002] Currently, due to the differences in the distribution of EEG signals among individuals, new users must collect a certain amount of EEG data for calibration when using non-invasive brain-computer interfaces to decode EEG signals. For example... Figure 2 As shown, existing technologies mainly improve similarity by reducing the distribution differences of EEG signals among different individuals, which is a method of extracting invariant features from between domains. EEG data acquisition is time-consuming and labor-intensive; excessive calibration time can easily cause brain fatigue and affect the quality of EEG signals.

[0003] Therefore, how to propose a low-duration calibration EEG decoding method and system based on invariant feature extraction to overcome the shortcomings of the existing technology is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the present invention provides a low-duration calibration EEG decoding method and system based on invariant feature extraction. Phase information is used as an intra-domain invariant feature to improve the similarity of EEG data from different individuals. By studying the similar features in the EEG signals of different individuals, invariant feature extraction between individuals is achieved, thereby reducing calibration time. To achieve the above objective, the present invention adopts the following technical solution: A low-duration calibration EEG decoding method based on invariant feature extraction includes: Acquire labeled EEG data, perform Fourier transform to obtain amplitude and phase information; Phase information and labels are input into the teacher network for training, and the feature output of the teacher network is obtained. Labeled EEG data are randomly paired and reconstructed to obtain reconstructed EEG data. The reconstructed EEG data is then input into a student network, which learns the phase information extracted from the teacher network to obtain the feature output of the student network. Real-time data is input into the student network to obtain EEG decoding and recognition results.

[0005] Optionally, performing the Fourier transform includes: For a single-channel EEG sample x, the Fourier transform calculation formula is: ,in, For sequence index, They are length and height, respectively. The indices are for length and height, respectively. Let j be the natural constant and j be the imaginary unit. The amplitude information is calculated as follows: ; The phase information is calculated as follows: ,in, This represents the amplitude information of EEG sample x. This represents the phase information of EEG sample x. After Fourier transform The imaginary part, After Fourier transform The real part.

[0006] Optionally, the teacher network adopts the EEGNet network structure, and the phase information and labels are input into the teacher network for training, and the classification loss between the network output and the real label is calculated.

[0007] Optionally, the EEGNet network structure includes: a convolutional layer, a batch normalization layer, a deep convolutional layer, an ELU activation function, an average pooling layer, a dropout layer, a batch normalization layer, a separable convolutional layer, a batch normalization layer, an ELU activation function, an average pooling layer, a dropout layer, a flatten layer, and a fully connected layer connected in sequence.

[0008] Optionally, it also includes training the network through error backpropagation, with the number of training iterations (epochs) being n, using the Adam optimizer.

[0009] Optionally, the step of randomly pairing and reconstructing the labeled EEG data to obtain the reconstructed EEG data includes: randomly pairing EEG samples from different individuals, performing Fourier spectrum transfer, exchanging their phase information, retaining amplitude information, and then obtaining the reconstructed EEG data through inverse Fourier transform.

[0010] Optionally, the reconstructed EEG data specifically includes: Where X' represents the reconstructed EEG data. This is the inverse Fourier transform. To control the hyperparameters of the enhancement force, For the amplitude information of the i-th sample of individual S1, For the amplitude information of the m-th sample of individual S2, The phase information of the i-th sample of individual S1.

[0011] Optionally, the student network adopts the EEGNet network structure to learn the phase information extracted from the teacher network and obtain the feature output of the student network.

[0012] Optional, also includes: The correlation alignment loss of EEG features of different individuals in the feature output of the student network is calculated and expressed as: ,in, For the relevant alignment loss, and The covariance matrix represents the EEG sample features of two different individuals S1 and S2, d is the dimension of the feature output of the student network, F is the Frobenius norm, and the loss function is used to extract inter-domain invariant features. ,in, The feature output after an individual S1's EEG sample is input into the student network. A column vector consisting entirely of 1s, with superscripts... This is the transpose operation for a matrix; Calculate the mean squared error loss between the feature outputs of the student network and the feature outputs of the teacher network. The calculation formula is: Where n is the number of samples and t is the sample index. Let t be the output feature of the student network for the t-th sample. Let be the output features of the teacher network for the t-th sample; Loss function of student network Represented as: ,in, For classification loss.

[0013] Optionally, a low-duration calibration EEG decoding system based on invariant feature extraction includes: Acquisition module: Used to acquire tagged EEG data; Preprocessing module: used to perform Fourier transform to obtain amplitude and phase information; Teacher network module: Used to input phase information and labels into the teacher network for training, and obtain the feature output of the teacher network; Student network module: It is used to randomly pair and reconstruct labeled EEG data to obtain reconstructed EEG data. The reconstructed EEG data is then input into the student network, which learns the phase information extracted from the teacher network to obtain the feature output of the student network. Recognition module: Used to acquire real-time data input into the student network to obtain EEG decoding and recognition results.

[0014] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a low-duration calibration EEG decoding method and system based on invariant feature extraction, which has the following beneficial effects: This invention proposes a low-duration calibration EEG decoding method based on invariant feature extraction, comprising: acquiring labeled EEG data, performing Fourier transform to obtain amplitude and phase information; inputting the phase information and labels into a teacher network for training to obtain the feature output of the teacher network; randomly pairing and reconstructing the labeled EEG data to obtain reconstructed EEG data, inputting the reconstructed EEG data into a student network, learning the phase information extracted by the teacher network through the student network to obtain the feature output of the student network; and acquiring real-time data and inputting it into the student network to obtain the EEG decoding and recognition result.

[0015] This invention proposes an EEG decoding method based on inter-domain and intra-domain invariant feature extraction. (1) By extracting similarity features between different individuals through invariant feature extraction, the classifier trained by this method has good generalization ability. When analyzing the EEG signals of new users, since the extracted features have commonalities, no recalibration is required, and it can be directly used for the EEG analysis of new subjects, significantly improving the efficiency of non-invasive brain-computer interfaces and the accuracy of EEG recognition. (2) It reduces the calibration time required for the EEG decoding system. Focusing on extracting invariant features between different individuals, the extracted features have universality in different individuals. The features extracted from the EEG data of new users also have this invariance, so no special calibration is required. This method can realize the plug-and-play function of non-invasive EEG devices, reduce the usage threshold, and improve the efficiency of brain-computer interfaces. (3) It integrates the EEG data of a large number of individuals, effectively improving the decoding accuracy of EEG. Experimental results show that the proposed method has a significant accuracy improvement on three widely used public datasets. On the BCICIV2A dataset, this method outperforms the classic EEGConfomer method by 1.77%; on the BCICIV2B dataset, it outperforms by 3.01%; and on the OpenBMI dataset, it outperforms by 5.87%. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 The diagram shows the structural framework of a low-duration calibration EEG decoding method based on invariant feature extraction provided by this invention.

[0018] Figure 2 A structural framework diagram of the prior art provided for this invention.

[0019] Figure 3 A schematic diagram of the EEGNet network structure provided by this invention.

[0020] Figure 4 This is a schematic diagram illustrating the principle of a low-duration calibration EEG decoding method based on invariant feature extraction provided by the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] This invention discloses a low-duration calibration EEG decoding method based on invariant feature extraction, comprising: Acquire labeled EEG data, perform Fourier transform to obtain amplitude and phase information; Phase information and labels are input into the teacher network for training, and the feature output of the teacher network is obtained. Labeled EEG data are randomly paired and reconstructed to obtain reconstructed EEG data. The reconstructed EEG data is then input into a student network, which learns the phase information extracted from the teacher network to obtain the feature output of the student network. Real-time data is input into the student network to obtain EEG decoding and recognition results.

[0023] Furthermore, the Fourier transform includes: For a single-channel EEG sample x, the Fourier transform calculation formula is: ,in, For sequence index, They are length and height, respectively. The indices are for length and height, respectively. Let j be the natural constant and j be the imaginary unit. The amplitude information is calculated as follows: ; The phase information is calculated as follows: ,in, This represents the amplitude information of EEG sample x. This represents the phase information of EEG sample x. After Fourier transform The imaginary part, After Fourier transform The real part.

[0024] Furthermore, the teacher network adopts the EEGNet network structure, and the phase information and labels are input into the teacher network for training, and the classification loss between the network output and the real label is calculated.

[0025] Furthermore, such as Figure 3 As shown, the EEGNet network structure includes: a convolutional layer, a batch normalization layer, a deep convolutional layer, an ELU activation function, an average pooling layer, a dropout layer, a batch normalization layer, a separable convolutional layer, a batch normalization layer, an ELU activation function, an average pooling layer, a dropout layer, a flatten layer, and a fully connected layer connected in sequence.

[0026] Furthermore, it also includes training the network through error backpropagation, with the number of training iterations being n epochs, and using the Adam optimizer.

[0027] Furthermore, the step of randomly pairing and reconstructing the labeled EEG data to obtain the reconstructed EEG data includes: randomly pairing EEG samples from different individuals, performing Fourier spectrum transfer, exchanging their phase information, retaining amplitude information, and then obtaining the reconstructed EEG data through inverse Fourier transform.

[0028] Furthermore, the reconstructed EEG data specifically includes: Where X' represents the reconstructed EEG data. This is the inverse Fourier transform. To control the hyperparameters of the enhancement force, For the amplitude information of the i-th sample of individual S1, For the amplitude information of the m-th sample of individual S2, The phase information of the i-th sample of individual S1.

[0029] Furthermore, the student network adopts the EEGNet network structure to learn the phase information extracted from the teacher network and obtain the feature output of the student network.

[0030] Furthermore, it also includes: The correlation alignment loss of EEG features of different individuals in the feature output of the student network is calculated and expressed as: ,in, For the relevant alignment loss, and The covariance matrix represents the EEG sample features of two different individuals S1 and S2, d is the dimension of the feature output of the student network, F is the Frobenius norm, and the loss function is used to extract inter-domain invariant features. ,in, The feature output after an individual S1's EEG sample is input into the student network. A column vector consisting entirely of 1s, with superscripts... This is the transpose operation for a matrix; Calculate the mean squared error loss between the feature outputs of the student network and the feature outputs of the teacher network. The calculation formula is: Where n is the number of samples and t is the sample index. Let t be the output feature of the student network for the t-th sample. Let be the output features of the teacher network for the t-th sample; Loss function of student network Represented as: ,in, For classification loss.

[0031] In a specific implementation, a low-duration calibration EEG decoding system based on invariant feature extraction includes: Acquisition module: Used to acquire tagged EEG data; Preprocessing module: used to perform Fourier transform to obtain amplitude and phase information; Teacher network module: Used to input phase information and labels into the teacher network for training, and obtain the feature output of the teacher network; Student network module: It is used to randomly pair and reconstruct labeled EEG data to obtain reconstructed EEG data. The reconstructed EEG data is then input into the student network, which learns the phase information extracted from the teacher network to obtain the feature output of the student network. Recognition module: Used to acquire real-time data input into the student network to obtain EEG decoding and recognition results.

[0032] In a specific implementation, a low-duration calibration EEG decoding method based on invariant feature extraction is proposed. This method performs EEG decoding based on inter-domain and intra-domain invariant feature extraction, such as... Figure 1 As shown, similarity features between different individuals are extracted using an invariant feature extraction method. The classifier trained using this method exhibits good generalization ability. When analyzing the EEG signals of new users, because the extracted features share commonalities, no further calibration is required, and the data can be directly used for EEG analysis of new subjects. This significantly improves the efficiency and accuracy of non-invasive brain-computer interfaces. A schematic diagram of the principle is shown below. Figure 4 As shown, the specific steps are as follows: Step 1: We have EEG data for N individuals, denoted as (X, Y), where X is the EEG data and Y is the corresponding label.

[0033] Fourier transforms were performed on these labeled raw EEG samples to obtain amplitude information A and phase information P. For a single-channel EEG sample x, its Fourier transform is calculated as follows: ,in, For sequence index, They are length and height, respectively. The indices are for length and height, respectively. is a natural constant, and j is the imaginary unit.

[0034] The amplitude information is calculated as follows: ; The phase information is calculated as follows: ,in, This represents the amplitude information of EEG sample x. This represents the phase information of EEG sample x. After Fourier transform The imaginary part, After Fourier transform The real part.

[0035] Step 2: Input the phase information P and the label Y into the teacher network for training. The teacher network adopts the EEGNet network structure, and the feature output Tf of the network is obtained. By training with the phase information, the features output by the network are made to include phase information.

[0036] This process extracts domain-invariant features. The network input is phase information P. The classification loss between the network output and the true label Y is calculated. The network is trained through error backpropagation, with 5000 epochs, using the Adam optimizer.

[0037] Step 3: Randomly pair EEG samples from different individuals, perform Fourier spectrum transfer, exchange their phase information P, retain the amplitude information A, and then obtain the reconstructed EEG data X' through inverse Fourier transform. The calculation formula is as follows: ,in, This is the inverse Fourier transform. The hyperparameter for controlling the enhancement force is in the range of (0-1). For the amplitude information of the i-th sample of individual S1, For the amplitude information of the m-th sample of individual S2, The phase information of the i-th sample of individual S1.

[0038] Step 4: The student network adopts the EEGNet network structure, and the feature output of this network is Sf. The student network is used to learn the phase information extracted by the teacher network.

[0039] (1) Calculate the correlation alignment loss of EEG features of different individuals in Sf, using the following formula: ,in, For the relevant alignment loss, and The covariance matrix representing the EEG sample features of two different individuals S1 and S2 is calculated as follows: d is the dimension of feature Sf. The EEG sample of individual S1 is input into the student network as a feature output, where F is the Frobenius norm. A column vector consisting entirely of 1s, with superscripts... This is the transpose of a matrix. This loss function is used to extract inter-domain invariant features.

[0040] (2) Calculate the mean square error loss between Sf and Tf The calculation formula is: Where n is the number of samples and t is the sample index. Let t be the output feature of the student network for the t-th sample. Let be the output features of the teacher network for the t-th sample.

[0041] (3) Loss function of student network Represented as ,in, For classification loss.

[0042] The resulting student network can be directly used for EEG data analysis of new subjects.

[0043] Step 5: The training data for the student network consists of reconstructed EEG data X' and corresponding labels Y, with the loss function being... The network was trained using backpropagation, with 200 training iterations, and the Adam optimizer was used.

[0044] In a specific implementation, step one converts existing EEG samples into phase information P, which, along with the corresponding label Y, is input into the teacher network. Step two then trains the teacher network to obtain a trained network. Step three obtains reconstructed EEG data X', which, along with the corresponding label Y, is input into the student network in step four to calculate the loss function. Step five trains the student network to obtain a trained network. When a new user needs EEG decoding, the user's EEG signal is directly collected and input into the trained student network for calculation to obtain the decoded EEG value.

[0045] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0046] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A low-duration calibration EEG decoding method based on invariant feature extraction, characterized in that, include: Acquire labeled EEG data, perform Fourier transform to obtain amplitude and phase information; Phase information and labels are input into the teacher network for training, and the feature output of the teacher network is obtained. Labeled EEG data are randomly paired and reconstructed to obtain reconstructed EEG data. The reconstructed EEG data is then input into a student network, which learns the phase information extracted from the teacher network to obtain the feature output of the student network. Real-time data is input into the student network to obtain EEG decoding and recognition results.

2. The low-duration calibration EEG decoding method based on invariant feature extraction according to claim 1, characterized in that, The Fourier transform includes: For a single-channel EEG sample x, the Fourier transform calculation formula is: ,in, For sequence index, They are length and height, respectively. The indices are for length and height, respectively. Let j be the natural constant and j be the imaginary unit. The amplitude information is calculated as follows: ; The phase information is calculated as follows: ,in, This represents the amplitude information of EEG sample x. This represents the phase information of EEG sample x. After Fourier transform The imaginary part, After Fourier transform The real part.

3. The low-duration calibration EEG decoding method based on invariant feature extraction according to claim 1, characterized in that, The teacher network adopts the EEGNet network structure. Phase information and labels are input into the teacher network for training, and the classification loss between the network output and the real label is calculated.

4. The low-duration calibration EEG decoding method based on invariant feature extraction according to claim 3, characterized in that, The EEGNet network structure includes, in sequence, a convolutional layer, a batch normalization layer, a deep convolutional layer, an ELU activation function, an average pooling layer, a dropout layer, a batch normalization layer, a separable convolutional layer, a batch normalization layer, an ELU activation function, an average pooling layer, a dropout layer, a flatten layer, and a fully connected layer.

5. The low-duration calibration EEG decoding method based on invariant feature extraction according to claim 4, characterized in that, It also includes training the network through error backpropagation, with n epochs of training iterations, using the Adam optimizer.

6. The low-duration calibration EEG decoding method based on invariant feature extraction according to claim 1, characterized in that, The process of randomly pairing and reconstructing labeled EEG data to obtain reconstructed EEG data includes: randomly pairing EEG samples from different individuals, performing Fourier spectrum transfer, exchanging their phase information, retaining amplitude information, and then obtaining reconstructed EEG data through inverse Fourier transform.

7. The low-duration calibration EEG decoding method based on invariant feature extraction according to claim 6, characterized in that, The reconstructed EEG data specifically includes: Where X' represents the reconstructed EEG data. This is the inverse Fourier transform. To control the hyperparameters of the enhancement force, For the amplitude information of the i-th sample of individual S1, For the amplitude information of the m-th sample of individual S2, The phase information of the i-th sample of individual S1.

8. The low-duration calibration EEG decoding method based on invariant feature extraction according to claim 1, characterized in that, The student network adopts the EEGNet network structure to learn the phase information extracted from the teacher network and obtain the feature output of the student network.

9. A low-duration calibration EEG decoding method based on invariant feature extraction according to claim 8, characterized in that, Also includes: The correlation alignment loss of EEG features of different individuals in the feature output of the student network is calculated and expressed as: ,in, For the relevant alignment loss, and The covariance matrix represents the EEG sample features of two different individuals S1 and S2, d is the dimension of the feature output of the student network, F is the Frobenius norm, and the loss function is used to extract inter-domain invariant features. ,in, The feature output after an individual S1's EEG sample is input into the student network. A column vector consisting entirely of 1s, with superscripts... This is the transpose operation for a matrix; Calculate the mean squared error loss between the feature outputs of the student network and the feature outputs of the teacher network. The calculation formula is: Where n is the number of samples and t is the sample index. Let t be the output feature of the student network for the t-th sample. Let be the output features of the teacher network for the t-th sample; Loss function of student network Represented as: ; For classification loss: ； in, The total number of categories, For category indexing, For the sample to belong to the first The probability of each category The model predicts that the sample belongs to the first... The probability of each category.

10. A low-duration calibration EEG decoding system based on invariant feature extraction, characterized in that, include: Acquisition module: Used to acquire tagged EEG data; Preprocessing module: used to perform Fourier transform to obtain amplitude and phase information; Teacher network module: Used to input phase information and labels into the teacher network for training, and obtain the feature output of the teacher network; Student network module: It is used to randomly pair and reconstruct labeled EEG data to obtain reconstructed EEG data. The reconstructed EEG data is then input into the student network, which learns the phase information extracted from the teacher network to obtain the feature output of the student network. Recognition module: Used to acquire real-time data input into the student network to obtain EEG decoding and recognition results.

Images