Electroencephalogram signal recognition method based on feature embedding enhancement and coupling

Abstract

The invention discloses an electroencephalogram signal recognition method based on feature embedding enhancement and coupling. The method comprises the following steps: firstly, filtering and normalizing collected electroencephalogram data, and denoising electroencephalogram signals so as to improve the signal quality; then capturing a potential characteristic mode in the signal by using operations such as data flipping, channel exchange and cutting; designing a neural network framework to perform feature extraction and coupling on the enhanced signal and the original signal; by means of learning and coupling of feature embedding, samples with similar neural modes can be gathered in a low-dimensional embedding space, meanwhile, samples with different neural modes are effectively separated, and therefore the accuracy and robustness of electroencephalogram signal classification are improved.

Description

Technical Field

[0001] The invention relates to a method for recognizing an electroencephalogram signal, and in particular to a method for extracting and embedding a feature of an electroencephalogram signal. Background Art

[0002] EEG signals are composed of electrical signals emitted by a large number of neurons, which can be used to decode action information. Brain-computer interface (BCI) technology can decode human brain activity and convert it into control instructions, thereby enabling the human brain to control external devices and promote human-computer interaction. Due to its potential application in promoting human-computer interaction and brain function recovery, brain-computer interface technology has received widespread attention.

[0003] Deep embedding learning aims to learn a discriminative low-dimensional feature embedding so that: 1) positive concentration, the embedding features of EEG signal samples of the same category are close to each other; 2) negative separation, the embedding features of EEG signal samples of different categories are distinguished. Supervised embedding learning has shown excellent performance in EEG signal recognition tasks. However, EEG data collection is difficult and the sample size is small. At the same time, sufficient labeled data required for supervised methods is usually difficult to obtain, because the labeling of EEG data requires a lot of expensive manpower and expertise in specific fields.

[0004] The main challenge of EEG embedding learning is to discover positive supervisory information from limited nonlinear EEG images and generalize to unknown categories. Existing methods usually distinguish positive and negative samples by labels, which greatly limits the size of the feature space. At the same time, this method relies heavily on the quality of the initial features and usually requires a pre-trained model, which makes the training as a whole separate and fails to fully utilize the feature extraction capabilities of the neural network.

[0005] Therefore, an efficient feature extraction method is needed to fully mine the information of different samples in EEG signals and improve the ability to distinguish and recognize EEG signals. At the same time, the method should have a certain degree of robustness and be able to adapt to different task requirements and data scenarios, providing more reliable support for EEG signal analysis and decoding.

[0006] After searching, the application publication number CN117814812A, a method for EEG signal recognition based on feature enhancement and difference amplification, includes the following steps: S1, constructing a corresponding component space filter according to the task type of the EEG signal data; S2, using the component space filter to filter the EEG signal data; S3, horizontally splicing the EEG signal data of the same task type that has been filtered to form enhanced EEG signal data; S4, using the common space pattern algorithm to enhance the EEG signal data for difference amplification and feature extraction, and inputting the features into the classifier for classification. Beneficial effects of the invention: The method proposed in the present invention can effectively extract the weak features of the MRCPs signal and obtain good classification accuracy by performing feature enhancement and difference amplification on the EEG signal data, and can be effectively applied to motion-related brain-computer interface systems.

[0007] This patent constructs component spatial filters according to the task type. These filters linearly combine the channels of the input signal and adjust these filters to maximize and minimize the signal variance of the two categories. It is difficult to capture complex nonlinear and high-order interaction features, and the adaptability is poor. Several types of spatial filters are required for several types of tasks. The present invention uses a deep learning framework and does not rely on the task type to classify samples, but instead uses the complex relationship between samples to effectively capture high-dimensional features. At the same time, as the task categories increase, the present invention only needs to modify the classification output without adding a feature extractor.

[0008] This patent horizontally splices the extracted features of the same category to form an enhanced EEG signal. The present invention uses feature coupling and multi-head attention mechanism to strengthen the correlation between different signals without changing the size of single sample data, and also avoids the feature redundancy problem of traditional horizontal splicing methods. Summary of the invention

[0009] The present invention aims to solve the problem of insufficient feature extraction caused by small data sets in the field of EEG signal recognition. A method for EEG signal recognition based on feature embedding enhancement and coupling is proposed. The technical solution of the present invention is as follows:

[0010] A method for recognizing electroencephalogram signals based on feature embedding enhancement and coupling comprises the following steps:

[0011] Step 1: Manually label the collected EEG signal data and preprocess them. The preprocessing includes bandpass filtering and normalization to eliminate noise and artifacts in the signal.

[0012] Step 2: Perform data enhancement on the preprocessed EEG signals by randomly using data flipping, channel swapping, and cropping methods;

[0013] Step 3: Construct a feature-enhanced coupled neural network, use inner product and edge network to explore the relationship between samples, use feature coupling module to optimize the network structure, and enhance the diversified expression and relevance of feature embedding;

[0014] Step 4: Concatenate the data in step 2 with the original data and train the feature-enhanced coupled neural network;

[0015] Step 5: Test the trained feature-enhanced coupled neural network. First, preprocess the test data in step 1, and then extract features through the feature-enhanced coupled neural network in step 3 and output the recognition results.

[0016] Furthermore, in step 1: the EEG signal is collected by a Brain Products 32-lead EEG cap, and four types of motor imagery signals of the subject are collected, namely, the imagination of moving the left hand, the right hand, both feet and the tongue; the signal is filtered by a Butterworth filter in the range of 0.5 Hz-40 Hz, the information related to the motor imagery is extracted, and the z score is used for normalization processing;

[0017] Furthermore, the step 2 performs data enhancement on the preprocessed EEG signal by randomly using data flipping, channel swapping and cropping methods, specifically including:

[0018] Data augmentation is to randomly select a certain number of samples from the entire data set in each training iteration, and apply random operations to expand the original data. For example, for a sample, there is a certain probability that the sample will be flipped or the channel will be swapped.

[0019] Furthermore, the step 3 specifically includes: constructing a feature enhancement coupled neural network and using a spatiotemporal convolutional neural network to realize feature extraction; first, the input data is m original EEG samples, and m enhanced samples are generated by data enhancement, and these samples are respectively sent to the same spatiotemporal convolutional neural network to extract features; the spatiotemporal convolutional network consists of a temporal convolution, a spatial convolution and a separable convolution; for each EEG sample x i There is a randomly augmented positive sample And 2m-2 negative samples, negative samples can increase the recognition x i and The difficulty is used to increase the recognition ability of the network;

[0020] Obtain the embedding feature f through the time channel convolutional neural network i and Then it goes through the feature coupling module; the coupling module is designed based on the graph neural network; a two-layer perceptron is used as the edge network to adaptively learn the topological structure of the feature space; f i and The absolute difference of the edge network is used as the input of the edge network, and then the output of the edge network and f i and The combination is the relationship matrix A; information coupling is achieved through the relationship matrix A; the coupling formula is as follows:

[0021]

[0022] f i The value of the cth channel in is replaced by The value of the cth channel in is one of the first k features matched according to the relationship matrix A, so such channel coupling will be performed k times. Similarly, the swapped features are passed through the same multi-head attention layer to obtain the embedded feature F i and Finally, it is fed into the classifier;

[0023] The constructed feature embedding network fully exploits the relationship between all samples sampled in a mini-batch.

[0024] Furthermore, the step 4 concatenates the data in step 2 with the original data and trains a feature-enhanced coupled neural network, specifically including:

[0025] When training the feature-enhanced coupled neural network, multiple loss functions are used to update the gradient of model training; the task loss function uses the cross entropy function, and its formula is:

[0026] loss cls =-Σp i logq i

[0027] In the formula, p represents the expected value and q represents the predicted value;

[0028] The feature space is spatially restricted, namely difference loss and self-similarity loss. The difference loss formula is as follows:

[0029]

[0030] In the formula, ‖.‖ represents the Frobenius norm;

[0031] The self-similarity loss formula is as follows:

[0032]

[0033] Where mix(·) represents minimization, N is the number of samples, and E is a unit matrix.

[0034] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for recognizing electroencephalogram signals based on enhanced feature embedding and coupling as described in any one of the above is implemented.

[0035] A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any one of the methods for recognizing electroencephalogram signals based on enhanced feature embedding and coupling.

[0036] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the method for recognizing electroencephalogram signals based on enhanced feature embedding and coupling as described in any one of the above is implemented.

[0037] The advantages and beneficial effects of the present invention are as follows:

[0038] Since the scale of EEG signal data is very small, feature extraction is insufficient. The present invention proposes an EEG signal recognition method based on feature embedding enhancement and coupling. The random enhancement strategy of step 2 enriches the feature space and effectively improves the generalization ability of EEG signal samples, laying a solid foundation for subsequent embedding learning and classification tasks. On top of this, the coupled neural network framework of step 3 jointly learns the original EEG signal and the enhanced signal, and after being constrained by multiple loss functions, the embedded features realize the alignment and coupling of neural patterns in the low-dimensional space. By aligning features in the low-dimensional embedding space, the overfitting problem of the high-dimensional space is avoided, and at the same time, the feature complementarity between different samples is promoted, further improving the classification accuracy. Unlike traditional data enhancement methods, the embedded learning strategy adopted in the present invention fully mines the information between EEG signal samples.

[0039] In addition, the present invention enhances the sensitivity to effective features through multi-head attention, highlights important features and weakens redundant features, and significantly improves the robustness of EEG signal recognition. The proposed method has low computational complexity and is easy to implement. It is not only suitable for EEG signal recognition, but can also be extended to other feature extraction tasks of small sample high-dimensional data, providing an efficient and innovative solution for the field of EEG decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The present invention provides a preferred embodiment of a method for electroencephalogram signal based on feature embedding enhancement and coupling.

[0041] Figure 2 Figure 2 is the structural diagram of the feature enhanced coupled neural network. DETAILED DESCRIPTION

[0042] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0043] The technical solution of the present invention to solve the above technical problems is:

[0044] As shown in the figure, this embodiment provides an EEG signal domain adaptation method based on Riemannian manifold coordinate alignment, comprising the following steps:

[0045] Step 1: EEG signals were collected through a Brain Products 32-lead EEG cap, and four types of motor imagery signals were collected from the subjects, namely, the imagination of moving the left hand, right hand, feet, and tongue. The signals were filtered with a Butterworth filter in the range of 0.5Hz-40Hz to extract information related to motor imagery, and normalized using z-scores.

[0046] Step 2: Data augmentation is to randomly select a certain number of samples from the entire data set in each training iteration and apply random operations to augment the original data. Take a sample as an example. There is a certain probability that the sample will be flipped or channel swapped.

[0047] Step 3: Construct a feature-enhanced coupled neural network and use a spatiotemporal convolutional neural network to extract features. First, the input data is m original EEG samples. Through data enhancement, m enhanced samples are generated. These samples are sent to the same spatiotemporal convolutional neural network to extract features. The spatiotemporal convolutional network consists of a temporal convolution, a spatial convolution, and a separable convolution. For each EEG sample x i There is a randomly augmented positive sample And 2m-2 negative samples, negative samples can increase the recognition x i and This can increase the network's recognition ability.

[0048] Obtain the embedding feature f through the time channel convolutional neural network i and Then it goes through the feature coupling module. The coupling module is designed based on the graph neural network. A two-layer perceptron is used as the edge network to adaptively learn the topological structure of the feature space. i and The absolute difference of the edge network is used as the input of the edge network, and then the output of the edge network and f i and The combination is the relationship matrix A. Information coupling is achieved through the relationship matrix A. The coupling formula is as follows:

[0049]

[0050] fi The value of the cth channel in is replaced by The value of the cth channel in is one of the first k features matched according to the relationship matrix A, so such channel coupling will be performed k times. Similarly, the swapped features are passed through the same multi-head attention layer to obtain the embedded feature F i and Finally it is fed into the classifier.

[0051] The constructed feature embedding network fully exploits the relationship between all samples sampled in a mini-batch.

[0052] Step 4: When training the feature-enhanced coupled neural network, multiple loss functions are used to update the gradient of model training. The task loss function uses the cross entropy function, and its formula is:

[0053] loss cls =-Σp i logq i

[0054] In the formula, p represents the expected value and q represents the predicted value.

[0055] The feature space is spatially restricted, namely difference loss and self-similarity loss. The difference loss formula is as follows:

[0056]

[0057] In the formula, ||.|| represents the Frobenius norm.

[0058] The self-similarity loss formula is as follows:

[0059]

[0060] Where mix(·) represents minimization, N is the number of samples, and E is a unit matrix.

[0061] Step 5: Use EEG data to test the classification performance of feature-enhanced coupled neural networks. First, extract discriminative low-dimensional feature representations through the neural network and feed them as input features to the improved fully connected classifier. The Softmax layer is used to calculate the confidence probability distribution of each category, thereby achieving accurate classification of EEG signals.

[0062] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions.

[0063] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0064] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0065] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A method for recognizing electroencephalogram signals based on feature embedding enhancement and coupling, characterized in that: The following steps are involved: Step 1: Manually label the collected EEG signal data and preprocess them. The preprocessing includes bandpass filtering and normalization to eliminate noise and artifacts in the signal. Step 2: Perform data enhancement on the preprocessed EEG signals by randomly using data flipping, channel swapping, and cropping methods; Step 3: Construct a feature-enhanced coupled neural network, use inner product and edge network to explore the relationship between samples, use feature coupling module to optimize the network structure, and enhance the diversified expression and relevance of feature embedding; Step 4: Concatenate the data in step 2 with the original data and train the feature-enhanced coupled neural network; Step 5: Test the trained feature-enhanced coupled neural network. First, preprocess the test data in step 1, and then extract features through the feature-enhanced coupled neural network in step 3 and output the recognition results.

2. According to the method for recognizing EEG signals based on feature embedding enhancement and coupling described in claim 1, it is characterized in that: The step 1: EEG signals are collected through a Brain Products 32-lead EEG cap, and four types of motor imagery signals of the subject are collected, namely, the imagination of moving the left hand, the right hand, both feet and the tongue; the signals are filtered with a Butterworth filter in the range of 0.5 Hz-40 Hz, the information related to motor imagery is extracted, and the z score is used for normalization processing;.

3. According to the method for recognizing EEG signals based on feature embedding enhancement and coupling as described in claim 1, it is characterized in that: The step 2 performs data enhancement on the preprocessed EEG signal by randomly using data flipping, channel swapping and cropping methods, specifically including: Data augmentation is to randomly select a certain number of samples from the entire data set in each training iteration, and apply random operations to expand the original data. For example, for a sample, there is a certain probability that the sample will be flipped or the channel will be swapped.

4. According to the method for recognizing EEG signals based on feature embedding enhancement and coupling as described in claim 1, it is characterized in that: The step 3 specifically includes: constructing a feature enhancement coupled neural network and using a spatiotemporal convolutional neural network to realize feature extraction; first, the input data is m original EEG samples, and m enhanced samples are generated by data enhancement. These samples are respectively sent to the same spatiotemporal convolutional neural network to extract features; the spatiotemporal convolutional network consists of a temporal convolution, a spatial convolution and a separable convolution; for each EEG sample x i There is a randomly augmented positive sample And 2m-2 negative samples, negative samples can increase the recognition x i and The difficulty is used to increase the recognition ability of the network; Obtain the embedding feature f through the time channel convolutional neural network i and Then it goes through the feature coupling module; the coupling module is designed based on the graph neural network; a two-layer perceptron is used as the edge network to adaptively learn the topological structure of the feature space; f i and The absolute difference of the edge network is used as the input of the edge network, and the output of the edge network and f i and The combination is the relationship matrix A; information coupling is achieved through the relationship matrix A; the coupling formula is as follows: f i The value of the cth channel in is replaced by The value of the cth channel in is one of the first k features matched according to the relationship matrix A, so such channel coupling will be performed k times. Similarly, the swapped features are passed through the same multi-head attention layer to obtain the embedded feature F i and Finally, it is fed into the classifier; The constructed feature embedding network fully exploits the relationship between all samples sampled in a mini-batch.

5. The method for EEG signal recognition based on enhanced feature embedding and coupling according to claim 1, characterized in that: Step 4 combines the data in step 2 with the original data. Training feature-enhanced coupled neural networks, including: When training the feature-enhanced coupled neural network, multiple loss functions are used to update the gradient of model training; the task loss function uses the cross entropy function, and its formula is: loss cls =-∑p i logq i In the formula, p represents the expected value and q represents the predicted value; The feature space is spatially restricted, namely difference loss and self-similarity loss. The difference loss formula is as follows: In the formula, ||.|| represents the Frobenius norm; The self-similarity loss formula is as follows: Where mix(·) represents minimization, N is the number of samples, and E is a unit matrix.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the electroencephalogram signal recognition method based on enhanced feature embedding and coupling as described in any one of claims 1 to 5 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for recognizing electroencephalogram signals based on enhanced feature embedding and coupling as described in any one of claims 1 to 5 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for recognizing electroencephalogram signals based on enhanced feature embedding and coupling as described in any one of claims 1 to 5 is implemented.

Images