A method for EEG signal classification based on collaborative contrast regularized average teacher model

Abstract

The present invention provides an electroencephalogram (EEG) signal classification method based on a collaborative contrast regularized average teacher model, comprising the following steps: firstly, electroencephalogram (EEG) signals are preprocessed and a two-stage EEG data augmentation method is used to generate EEG data under a new perspective, and EEG data under another perspective is generated in the time-frequency domain; the student and teacher networks in the average teacher model are used to learn the EEG data features of different perspectives respectively; the consistency loss is used to encourage the consistency of the prediction of the student and teacher networks for the EEG data of different perspectives, and the collaborative contrast loss is used to encourage the consistency of the EEG data structure of different perspectives, and finally the cross-entropy loss is weighted and summed to optimize the network parameters. The two-stage EEG data augmentation method proposed in the present invention can simultaneously capture the change factors of the time domain and the time-frequency domain, and the proposed collaborative contrast regularized average teacher model can learn the data level information under different perspectives and the consistency information of the data structure between different perspectives, thereby improving the robustness, discrimination and generalization of the model.

Description

Technical Field

[0001] The invention relates to an electroencephalogram (EEG) signal recognition method based on a collaborative contrast regularized average teacher model, and belongs to the field of EEG signal recognition. Background Art

[0002] Brain Computer Interface (BCI) refers to the creation of a direct connection between a person or animal and an external device to achieve information exchange between the brain and the device. At present, BCI systems have been widely used to help patients with stroke, spinal cord injury, etc. control external devices to improve their quality of life. Electroencephalogram (EEG) is a bioelectric phenomenon generated by the transmission of information between brain cell groups in the form of ions. It is a general reflection of the electrophysiological activity of neurons in the cerebral cortex or scalp surface. Compared with other types of brain signals, EEG has the advantages of convenient acquisition and high spatiotemporal resolution, and has unique application value in BCI systems. With the help of a large amount of EEG training data, deep learning methods have greatly improved the recognition performance of EEG. However, in addition to the common problems of stability and low signal-to-noise ratio in EEG in some BCI system paradigms (such as motor imagery EEG), there is also the problem that it is difficult to obtain a large amount of training data due to the time-consuming and labor-intensive data acquisition. EEG data augmentation methods can alleviate the above-mentioned problem of insufficient training data. EEG contains rich information in time, frequency, and space, and can be represented by features in multiple domains (such as time domain, time-frequency domain, etc.). However, existing EEG data augmentation methods usually generate new augmented data in a single domain, ignoring the variation factors of EEG data in multiple domains. The quality of augmented EEG data is not high, which brings challenges to the establishment of a robust EEG recognition model. In addition, most of the current deep learning methods only consider the information at the EEG data level under different perspectives, ignoring the information at the EEG data structure level that is unchanged between different perspectives, making it difficult to break through the performance bottleneck of the EEG recognition model.

[0003] In response to the above problems, the present invention relies on the existing deep learning methods, firstly improves the EEG data augmentation mechanism, establishes a new two-stage EEG data augmentation mechanism, realizes the augmented representation of EEG data in multiple domains, ensures that the generated EEG data can simultaneously capture the change factors of multiple domains, and enhances the generalization performance of the existing model; then, the present invention constructs a collaborative constraint method between multi-view EEG data structures, learns the intra-class similarity and inter-class difference information at the EEG data structure level through contrast regularization, and encourages the consistency of EEG data structures between different perspectives through collaborative constraints, further improving the model's classification performance of EEG data. Summary of the invention

[0004] The present invention provides an EEG signal recognition method based on a collaborative contrast regularized average teacher model. A two-stage EEG data augmentation method is adopted to generate EEG data that can simultaneously capture the change factors of multiple domains. On this basis, an average teacher network model including student and teacher networks is introduced to learn EEG deep features under different perspectives. A consistency loss function is used to encourage the consistency of the prediction of EEG data under different perspectives by the student and teacher network models, and a collaborative contrast loss function is used to encourage the consistency of the EEG data structure under different perspectives. Finally, the cross entropy loss function, the consistency loss function and the collaborative contrast loss function are weighted and summed to form a final loss function to optimize the network parameters.

[0005] The present invention adopts the following technical solutions to solve the above problems:

[0006] A method for recognizing electroencephalogram signals based on a collaborative contrast regularized average teacher model comprises the following steps:

[0007] Step 1: Collect EEG data.

[0008] Step 2: Preprocess the EEG data, including bandpass filtering and artifact removal, and select specific time periods and frequency bands to obtain EEG data in represents the nth EEG data, c and d represent the number of electrode channels and time sampling points of the EEG data respectively, and y n ∈{0, 1, 2, 3} is the label corresponding to the nth EEG data.

[0009] Step 3: Use different data augmentation methods to segment and reorganize the EEG data to obtain augmented EEG data from different perspectives and in It is the EEG data obtained by two-stage augmentation.

[0010] Step 4: Use the dedicated EEG decoding network as the backbone network of the average teacher model to transform the augmented EEG data And the EEG data after swapping the order Input the student network and teacher network in the average teacher model respectively.

[0011] Step 5: Use the probability output obtained by the softmax function of the output of the student network in step 4 together with the true label to calculate the classification loss.

[0012] Step 6: Use the outputs of the student network and the teacher network in step 4 to obtain their respective probability outputs through the softmax function, and calculate the consistency loss between the two.

[0013] Step 7: Use the projection network to reduce the dimension of the EEG features of different perspectives output in step 4, and then calculate the collaborative contrast loss between the features after dimension reduction at each perspective.

[0014] Step 8: The classification loss, consistency loss and collaborative contrast loss are weighted and summed to form the final loss function, and the back propagation algorithm is used to optimize the entire network.

[0015] Preferably, the EEG data augmentation method proposed in the present invention includes two stages. First, linear interpolation technology is introduced into the segmentation and reconstruction method in the EEG time domain. Secondly, the time domain EEG augmented data is further converted into its frequency domain representation through short-time Fourier transform. After the EEG signal is segmented and reorganized in the time-frequency domain, it is converted back into a time domain EEG signal using an inverse short-time Fourier transform as augmented EEG data.

[0016] Preferably, the collaborative contrast regularized average teacher model uses the collaborative contrast regularization mechanism as a core module, which is specifically expressed as follows:

[0017]

[0018] in,

[0019] in, and They represent the results obtained after the features of EEG data learned by the student (teacher) network under different augmentation methods are passed through the projection network and normalized. represents all samples except the i-th sample, P(i)≡{p∈A(i):y p =y i} indicates that the label y p The same set of all samples, τ represents the scale parameter.

[0020] Beneficial effects:

[0021] 1. The present invention introduces a two-stage EEG data augmentation method. In the first stage, linear interpolation technology is used in the time domain of EEG signals to avoid the problem of high-frequency noise brought by traditional segmentation and recombination methods to augment EEG data. In the second stage, the augmented EEG data obtained in the first stage are further augmented by segmentation and recombination in the time-frequency domain of EEG signals. The finally generated EEG data can simultaneously capture the changing factors in the time domain and the time-frequency domain, thereby enhancing the robustness and generalization ability of the model.

[0022] 2. Currently, the average teacher model can only learn information at the EEG data level. The present invention introduces a collaborative contrast regularization loss for the average teacher model. On the one hand, the contrast regularization constraint is used to ensure that the normalized embedding of the EEG data has better intra-class compactness and inter-class separability, thereby learning the EEG data structure information; on the other hand, the design of a collaborative learning mechanism can learn the consistency information of the EEG data structure from different perspectives, thereby improving the discriminative learning ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a network framework diagram of the EEG signal recognition method based on the collaborative contrast regularized average teacher model in the present invention. DETAILED DESCRIPTION

[0024] See also Figure 1 As shown:

[0025] The present invention is further explained below in conjunction with examples.

[0026] The main implementation process of the present invention is as follows, and the relevant network framework is shown in Figure 1 .

[0027] Step 1: Collect EEG data.

[0028] Step 2: Preprocess the EEG data, including bandpass filtering and artifact removal, and select specific time periods and frequency bands to obtain EEG data in represents the nth EEG data, c and d represent the number of electrode channels and time sampling points of the EEG data respectively, and y n ∈{0, 1, 2, 3} is the label corresponding to the nth EEG data.

[0029] Step 3: Segment and reorganize the EEG data according to two different data augmentation methods to obtain EEG data from different perspectives after augmentation and The specific EEG data augmentation methods are as follows:

[0030] 1) Use short-time Fourier transform on the preprocessed EEG data to convert it from time domain representation to time-frequency domain representation; use segmentation and recombination to first divide the EEG samples represented in the time-frequency domain into several non-overlapping segments, then randomly select other EEG samples with the same label according to the label of the current EEG sample, and obtain the time-frequency domain representation of the augmented sample by randomly combining the segments at the same position of different samples. Use inverse short-time Fourier transform to convert the newly generated EEG samples to time domain representation, that is, to form the augmented data corresponding to the EEG data, which is recorded here as

[0031] 2) Divide the preprocessed EEG data into J non-overlapping segments, and randomly select two segments from the EEG samples with the same label for interpolation fitting. The specific method is as follows:

[0032]

[0033] in, represents the current EEG sample, represents randomly selected EEG samples with the same label, R ij represents any integer randomly selected from [1, N], α is the interpolation fitting coefficient, and its value is 0.999. Then, using the method in 1) above, we can obtain two-stage augmented EEG data, which is recorded as

[0034] Step 4: Use the EEG decoding network Shallow ConvNet as the backbone network of the average teacher model to convert the augmented EEG data And the EEG data after changing the order Input the student network and teacher network in the average teacher model respectively.

[0035] Step 5: Use the probability output obtained by the softmax function of the output of the student network in step 4 together with the true label to calculate the classification loss. The formula is as follows:

[0036]

[0037] in, represents the classification loss, and They represent the augmented EEG data obtained using the two methods in step 3, respectively. i represents the true label of the sample, Represents the prediction results of the student network.

[0038] Step 6: Use the outputs of the student network and the teacher network in step 4 to obtain their respective probability outputs through the softmax function, and calculate the consistency loss between the two. The formula is as follows:

[0039]

[0040] in, represents the consistency loss, and They represent the augmented EEG data obtained using the two methods in step 3, and Represent the prediction results of the student network and the teacher network respectively.

[0041] Step 7: Use a projection network with the same structure to reduce the dimensionality of the EEG features of different perspectives learned in step 4, extract deeper EEG features, and use the outputs of the two projection networks to calculate the collaborative contrast loss between them. The formula is as follows:

[0042]

[0043]

[0044] in, and Represents the augmented EEG data in the current processing batch (Contains B EEG samples) The result after the EEG features learned by the student network and the teacher network are projected and normalized. represents all samples except the i-th sample in the current processing batch, P(i)≡{p∈A(i):y p =y i} indicates that the label y p The set of all the same positive samples, τ represents a scale parameter, and the value of the present invention is 0.07.

[0045] Step 8: The classification loss, consistency loss and collaborative contrast loss are weighted and summed to form the final loss function, as follows:

[0046]

[0047] Wherein, λ1 and λ2 represent the coefficients of the consistency loss function and the collaborative contrast loss function respectively, and the values ​​of the present invention are 1 and 0.001. The back propagation algorithm is used to optimize the student network to obtain the tth student network update parameter θ s , and the teacher network parameter θ t Depends on the student network parameters θ s , updated by the exponential moving average method, the formula is as follows:

[0048] θ t =β·θ t +(1-β)·θ s (9)

[0049] Wherein, β represents an update parameter, and the value of the present invention is 0.999.

[0050] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in the relevant technical field, are also included in the patent protection scope of the present invention.

Claims

1. A method for classifying EEG signals based on a collaborative contrast regularized average teacher model, characterized in that: Here are the steps: Step 1: Collect EEG data; Step 2: Preprocess the EEG data, including bandpass filtering and artifact removal, selecting specific time periods and frequency bands, and obtaining raw EEG data Where X n represents the nth EEG data, c, d represent the number of electrode channels and time sampling points of EEG signals, respectively, y n ∈{0, 1, 2, 3} is the label corresponding to the nth EEG data; Step 3: Use two data augmentation methods to segment and reorganize the EEG data to obtain augmented EEG data from different perspectives and in It is the EEG data obtained by two-stage augmentation method; Step 4: Use the EEG decoding network Shallow ConvNet as the backbone network of the average teacher model to And the EEG data after swapping the order Input the student network and teacher network in the average teacher model respectively; Step 5: Use the probability output obtained by the softmax function of the output of the student network in step 4 and the true label to calculate the classification loss; Step 6: Use the outputs of the student network and the teacher network in step 4 to obtain their respective probability outputs through the softmax function, and calculate the consistency loss between the two. The specific calculation method is: in, represents the consistency loss, and They represent the augmented EEG data obtained using the two methods in step 3, and Represent the prediction results of the student network and the teacher network respectively; Step 7: Use the projection network to reduce the dimension of the EEG features of different viewpoints output in step 4, and then calculate the collaborative contrast loss between the features after dimension reduction at each viewpoint. The specific calculation method is: In formula (3), and Represents the augmented EEG data in the current processing batch The result after the EEG features learned by using the student network s and the teacher network are projected and normalized is Contains B EEG samples, represents all samples except the i-th sample in the current processing batch, P(i)≡{A(i):y p =y i } indicates that the label y i The same set of all positive samples; τ represents a scale parameter with a value of 0.07; Step 8: The classification loss, consistency loss and collaborative contrast loss are weighted and summed to form the final loss function, and the back propagation algorithm is used to optimize the entire network.

2. The method for classifying EEG signals based on the collaborative contrast regularized average teacher model according to claim 1, characterized in that: In step 3, the two augmentation methods mainly include the following: 1) Use short-time Fourier transform to convert the preprocessed EEG data from time domain representation to time-frequency domain representation; adopt segmentation and recombination method to first divide the EEG samples represented in the time-frequency domain into several non-overlapping segments, then randomly select other EEG samples with the same label according to the label of the current EEG sample, and obtain the time-frequency domain representation of the augmented sample by randomly combining the segments at the same position of different samples; use inverse short-time Fourier transform to convert the newly generated EEG samples to time domain representation, that is, to form the augmented data corresponding to the EEG data, which is recorded here as 2) Divide the preprocessed EEG data into J non-overlapping segments, and randomly select two segments from the EEG samples with the same label for interpolation fitting. The specific method is as follows: in, represents the current EEG sample, represents randomly selected EEG samples with the same label, R ij represents any integer randomly selected from [1, N], α is the interpolation fitting coefficient, which is 0.

999. Then, using the method in 1) above, we can obtain two-stage augmented data, which is recorded as 3. The method for classifying EEG signals based on the collaborative contrast regularized average teacher model according to claim 1, characterized in that: In step 5, the probability output obtained by the softmax function of the output of the student network is used together with the true label to calculate the classification loss. The specific calculation method is: in, represents the classification loss, and They represent the augmented EEG data obtained using the two methods in step 3, respectively. i represents the true label of the sample, Represents the prediction results of the student network.

4. The method for classifying EEG signals based on the collaborative contrast regularized average teacher model according to claim 3, characterized in that: In step 8, the classification loss, consistency loss and collaborative contrast loss are weighted and summed to form the final loss function and the back propagation algorithm is used to optimize the entire network, as follows: Among them, λ1 and λ2 represent the coefficients of the consistency loss function and the collaborative contrast loss function, respectively, with values ​​of 1 and 0.

001. The back propagation algorithm is used to optimize the student network to obtain the tth student network update parameter θ s , and the teacher network parameter θ t Depends on the student network parameters θ s , updated by the exponential moving average method, the formula is as follows: i t =β·θ t +(1-β)·θ s (7) Among them, β represents the update parameter and its value is 0.999.

Images