An electroencephalogram signal recognition method based on feature embedding enhancement and coupling

Abstract

The application claims a kind of electroencephalogram recognition method based on feature embedding enhancement and coupling.First, the collected electroencephalogram data is filtered and normalized to denoise electroencephalogram signal, to improve signal quality;Then use data flip, channel exchange and cutting and other operations to capture potential feature patterns in the signal;Then a neural network framework is designed to extract and couple the enhanced signal and the original signal;Through the learning and coupling of feature embedding, the application can gather samples with similar neural patterns in a low-dimensional embedding space, while effectively separating samples with different neural patterns, thereby improving the accuracy and robustness of electroencephalogram classification.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of electroencephalogram signal recognition, and in particular to the field of feature extraction and embedding learning methods for electroencephalogram signals. BACKGROUND

[0002] Electroencephalogram signals are composed of electrical signals emitted by a large number of neurons, and these signals can be used to decode motor information. Brain-computer interface (BCI) technology can decode human brain activity and convert it into control instructions, thereby enabling brain control of external devices and facilitating human-computer interaction. Due to its potential applications in promoting human-computer interaction and brain function recovery, brain-computer interface technology has received extensive attention.

[0003] Deep embedding learning aims to learn a discriminative low-dimensional feature embedding such that: 1) positive concentration, the embedding features of electroencephalogram signal samples of the same class are close to each other; 2) negative separation, the embedding features of electroencephalogram signal samples of different classes are distinguished. Supervised embedding learning has shown excellent performance in electroencephalogram signal recognition tasks. However, electroencephalogram data acquisition is difficult with small sample size, and sufficient labeled data required by supervised methods are usually difficult to obtain because labeling of electroencephalogram data requires a large amount of expensive human labor and specific domain expertise.

[0004] The main challenge of electroencephalogram embedding learning is to discover positive supervision information from limited nonlinear electroencephalogram maps and the generalization ability to unknown classes. Existing methods usually distinguish positive samples and negative samples by labels, which greatly limits the size of the feature space. At the same time, this method largely depends on the quality of the initial features, usually requiring a pre-trained model, which separates the training as a whole and fails to fully utilize the feature extraction capability of neural networks.

[0005] Therefore, there is a need for an efficient feature extraction method that can fully exploit the information of different samples in electroencephalogram signals to improve the discrimination ability and recognition accuracy of electroencephalogram signals. At the same time, this method should have certain robustness to adapt to different task requirements and data scenarios, providing more reliable support for electroencephalogram signal analysis and decoding.

[0006] After retrieval, application publication number CN117814812A, a brain electrical signal recognition method based on feature enhancement and difference amplification, includes the following steps: S1, constructing a corresponding component space filter according to the task type of the brain electrical signal data; S2, filtering the brain electrical signal data using the component space filter; S3, transversely splicing the brain electrical signal data of the same task type after filtering to form enhanced brain electrical signal data; S4, using the common space mode algorithm to enhance the brain electrical signal data for difference amplification and feature extraction, and inputting the features into the classifier for classification. The beneficial effects of the present application are: the method proposed in the present application can effectively extract the weak features of MRCPs signals and obtain good classification accuracy by performing feature enhancement and difference amplification on brain electrical signal data, and can be effectively applied to motion-related brain-computer interface systems.

[0007] The patent constructs component space filters according to task types, which linearly combine each channel of the input signal. By adjusting these filters, the variance of the signal of the two classes is maximized and minimized. It is difficult to capture complex nonlinear and high-order interaction features, and the adaptability is poor. Several classes of tasks require several classes of spatial filters. The present application uses a deep learning framework to effectively capture high-dimensional features by utilizing the complex relationship between samples, rather than relying on task types to classify samples. At the same time, as the number of task categories increases, the present application only needs to modify the classification output, without the need to increase the feature extractor.

[0008] The patent transversely splices the extracted features of the same class to form enhanced brain electrical signals. The present application uses feature coupling and multi-head attention mechanisms to strengthen the correlation between different signals without changing the size of individual sample data, and also avoids the feature redundancy problem of traditional transverse splicing methods. SUMMARY

[0009] The present application aims to solve the problem of insufficient feature extraction caused by small data sets in the field of brain electrical signal recognition. A brain electrical signal recognition method based on feature embedding enhancement and coupling is proposed. The technical solution of the present application is as follows:

[0010] A brain electrical signal recognition method based on feature embedding enhancement and coupling includes the following steps:

[0011] Step 1: manually mark the class and preprocess the collected brain electrical signal data. Preprocessing includes bandpass filtering and normalization to eliminate noise and artifacts in the signal;

[0012] Step 2: data augmentation is performed on the preprocessed brain electrical signal, and random data flipping, channel exchange and cropping methods are used;

[0013] Step 3: build a feature enhancement coupling neural network, use inner product and edge network to explore the relationship between samples, use feature coupling module to optimize the network structure, enhance the diversity of feature embedding and correlation;

[0014] Step 4: splice the data in step 2 with the original data, and train the feature enhancement coupling neural network;

[0015] Step 5: test the trained feature enhancement coupling neural network, first preprocess the test data according to step 1, then extract features and output recognition results through the feature enhancement coupling neural network of step 3.

[0016] Further, the step 1: the electroencephalogram signal is collected by Brain Products 32 lead electroencephalogram cap, four types of motor imagination signals of the subject are collected, which are left hand, right hand, both feet and tongue imagination; The signal is filtered by Butterworth filter in the range of 0.5Hz-40Hz, the information related to motor imagination is extracted, and z-score is used for normalization processing;

[0017] Further, the step 2 is to perform data enhancement on the preprocessed electroencephalogram signal, and randomly use data flipping, channel exchange and cropping method, which specifically includes:

[0018] Data enhancement is to randomly select a certain number of samples from the entire data set in each training iteration, and apply random operations to expand the original data. Taking a sample as an example, the sample has a certain probability of being flipped or channel exchanged.

[0019] Further, the step 3 specifically includes: building a feature enhancement coupling neural network, using a spatio-temporal convolutional neural network to realize feature extraction; first, the input data is m original electroencephalogram samples, which are generated by data enhancement to generate m enhanced samples, which are sent into the same spatio-temporal convolutional neural network to extract features; The spatio-temporal convolutional network is composed of a time convolution, a space convolution and a separable convolution; for each electroencephalogram sample x i There is a randomly augmented positive sample and 2m-2 negative samples, the negative samples can increase the difficulty of identifying x i and , which is used to increase the recognition ability of the network;

[0020] Through the time channel convolutional neural network, the embedding feature f i and is obtained, and then the feature coupling module is passed; the coupling module is designed based on the graph neural network; a two-layer perception machine is used as an edge network to adaptively learn the topological structure of the feature space; f i and The absolute difference value of f i and The combination is the relationship matrix A; information coupling is realized through the relationship matrix A; the coupling formula is as follows:

[0021]

[0022] The value of the cth channel in f i is replaced by the value of the cth channel in f is one of the first k features matched according to the relationship matrix A, so such channel coupling will be performed k times, Similarly; the exchanged features are subjected to the same multi-head attention layer to obtain embedded features F i and Finally, it is sent to the classifier.

[0023] The constructed feature embedding network fully utilizes the relationship between all samples in small batch sampling.

[0024] Further, the step 4 splices the data in step 2 with the original data, and trains a feature enhancement coupling neural network, specifically including:

[0025] When training the feature enhancement coupling neural network, multiple loss functions are adopted for gradient update of model training; the task loss function adopts a cross-entropy function, and its formula is as follows:

[0026] loss cls =-Σp i logq i

[0027] In the formula, p represents an expected value, and q represents a predicted value.

[0028] The feature space is subjected to spatial restriction, which is difference loss and self-similarity loss, and the difference loss formula is as follows:

[0029]

[0030] In the formula, ‖.‖ represents the Frobenius norm.

[0031] The self-similarity loss formula is as follows:

[0032]

[0033] In the formula, mix(·) represents minimization, N is the number of samples, and E is a unit matrix.

[0034] ​An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, which, when executed by the processor, implements the method for recognizing electroencephalogram signals based on enhanced feature embedding and coupling according to any one of the preceding claims.

[0035] A non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the method for recognizing electroencephalogram signals based on enhanced feature embedding and coupling according to any one of the preceding claims.

[0036] A computer program product comprising a computer program, which, when executed by a processor, implements the method for recognizing electroencephalogram signals based on enhanced feature embedding and coupling according to any one of the preceding claims.

[0037] Advantages and beneficial effects of the present application are as follows:

[0038] Due to the small size of the electroencephalogram signal data, the feature extraction is insufficient, and the present application proposes a method for recognizing electroencephalogram signals based on feature embedding enhancement and coupling. The random enhancement strategy of step 2 enriches the feature space and effectively improves the generalization ability of the electroencephalogram signal sample, laying a solid foundation for subsequent embedding learning and classification tasks. On this basis, the coupling neural network framework of step 3 jointly learns the original electroencephalogram signal and the enhanced signal, and through the constraint of multiple loss functions, the embedding features realize the alignment and coupling of neural patterns in the low-dimensional space. By aligning the features in the low-dimensional embedding space, the overfitting problem in the high-dimensional space is avoided, and the feature complementarity between different samples is promoted, further improving the classification accuracy. Unlike traditional data enhancement methods, the embedding learning strategy adopted by the present application fully excavates the information between electroencephalogram signal samples.

[0039] In addition, the present application enhances the sensitivity to effective features through multi-head attention, highlights important features and weakens redundant features, significantly improving the robustness of electroencephalogram signal recognition. The proposed method has low computational complexity and is easy to implement, not only suitable for electroencephalogram signal recognition, but also applicable to feature extraction tasks of other small sample high-dimensional data, providing an efficient and innovative solution for the field of electroencephalogram decoding. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 A method flowchart for recognizing electroencephalogram signals based on feature embedding enhancement and coupling according to the preferred embodiment of the present application.

[0041] Figure 2 A feature enhancement coupling neural network structure diagram. DETAILED DESCRIPTION

[0042] With reference to the accompanying drawings: brief descriptions of the drawings

[0043] The technical scheme solving the above technical problems of the present application is:

[0044] As shown in the figure, the EEG signal domain adaptive method based on Riemannian manifold coordinate alignment provided by the embodiment comprises the following steps:

[0045] Step 1: The EEG signal is collected by a Brain Products 32-lead EEG cap, and four types of motor imagery signals of the subject are collected, which are the imagination of moving the left hand, the right hand, the feet and the tongue. The signal is filtered by a Butterworth filter in the range of 0.5Hz-40Hz, the information related to motor imagery is extracted, and the z-score is used for normalization processing

[0046] Step 2: Data augmentation is to randomly select a certain number of samples from the entire data set in each training iteration, and apply random operations to expand the original data. Taking a sample as an example. The sample has a certain probability of being flipped or channel exchanged.

[0047] Step 3: Construct a feature enhancement coupled neural network, and use a spatio-temporal convolutional neural network to realize feature extraction. First, the input data is m original EEG samples, which are generated by data augmentation to generate m enhanced samples, which are respectively sent into the same spatio-temporal convolutional neural network to extract features. The spatio-temporal convolutional network is composed of a time convolution, a space convolution and a separable convolution. For each EEG sample x i There is a randomly augmented positive sample and 2m-2 negative samples, the negative samples can increase the difficulty of identifying x i and , so as to increase the recognition ability of the network.

[0048] The embedding feature f i and is obtained through the time channel convolutional neural network, and then passes through the feature coupling module. The coupling module is designed based on the graph neural network. A two-layer perceptron is used as the edge network to adaptively learn the topological structure of the feature space. The absolute difference between f i and is taken as the input of the edge network, and the output of the edge network is combined with f i and to obtain the relationship matrix A. The information coupling is realized through the relationship matrix A. The coupling formula is as follows:

[0049]

[0050] fi The value of the cth channel is replaced with The value of the cth channel, Is one of the top k features matched according to the relationship matrix A, so such channel coupling will be performed k times, Similarly. The exchanged features are passed through the same multi-head attention layer to obtain the embedding features F i And Finally, it is sent to the classifier.

[0051] The constructed feature embedding network fully utilizes the relationship between all samples in the small batch sampling.

[0052] Step 4: When training the feature enhancement coupling neural network, multiple loss functions are used for gradient update of model training. The task loss function uses the cross-entropy function, and its formula is:

[0053] Loss cls = -Σp i Log q i

[0054] In the formula, p represents the expected value, and q represents the predicted value.

[0055] The feature space is spatially limited, which is difference loss and self-similarity loss, and the difference loss formula is as follows:

[0056]

[0057] In the formula, ||.|| represents the Frobenius norm.

[0058] The self-similarity loss formula is as follows:

[0059]

[0060] In the formula, mix(·) represents minimization, N is the number of samples, and E is a unit matrix.

[0061] Step 5: Test the classification performance of the feature enhancement coupling neural network using electroencephalogram data. First, extract discriminative low-dimensional feature representations through the neural network, and feed them as input features to the improved fully connected classifier. Use the Softmax layer to calculate the confidence probability distribution of each class, so as to realize accurate classification of electroencephalogram signals.

[0062] The system, device, module or unit illustrated in the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain functions.

[0063] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0064] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed, or inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0065] The above embodiments should be understood as merely illustrative of the present application and not restrictive of the scope of protection of the present application. After reading the description of the present application, those skilled in the art can make various modifications or changes to the present application, and these equivalent changes and modifications also fall within the scope defined by the claims of the present application.

Claims

1. A method for EEG signal recognition based on feature embedding enhancement and coupling, characterized in that, Includes the following steps: Step 1: Manually label and preprocess the acquired EEG signal data. Preprocessing includes bandpass filtering and normalization to eliminate noise and artifacts in the signal. Step 2: Perform data augmentation on the preprocessed EEG signals, randomly employing data flipping, channel swapping, and pruning methods; Step 3: We construct a feature-enhanced coupled neural network, use inner product and edge network to explore the relationship between samples, and use feature coupling module to optimize the network structure to enhance the diversity and correlation of feature embedding. Step 4: Concatenate the data from Step 2 with the original data to train the feature-enhanced coupled neural network; Step 5: Test the trained feature-enhanced coupled neural network. First, perform the preprocessing of the test data as in Step 1, and then extract features and output the recognition results through the feature-enhanced coupled neural network in Step 3. Step 3 specifically includes: constructing a feature enhancement coupled neural network, and using a spatiotemporal convolutional neural network to achieve feature extraction; firstly, the input data consists of m original EEG samples, which are augmented to generate m enhanced samples, and these samples are respectively fed into the same spatiotemporal convolutional neural network to extract features; the spatiotemporal convolutional network consists of a temporal convolution, a spatial convolution, and a separable convolution; for each EEG sample x i Each has a randomly augmented positive sample. And 2m-2 negative samples, negative samples can increase the recognition of x i and The difficulty is used to increase the network's recognition ability; Embedded features f are obtained through a time-channel convolutional neural network. i and Then it goes through the feature coupling module; The coupling module is designed based on a graph neural network; it utilizes a two-layer perceptron as the topology of the side network for adaptive learning of the feature space; f i and The absolute difference is used as the input to the edge network, and then the output of the edge network is summed with f. i and The combination is represented by relation matrix A; information coupling is achieved through relation matrix A; the coupling formula is as follows: f i The value of channel c is replaced with The value of the c-th channel, It is one of the first k features matched based on the relation matrix A, so this channel coupling will be executed k times. Similarly, the swapped features are passed through the same multi-head attention layer to obtain the embedded features F. i and Finally, it is fed into the classifier; The constructed feature embedding network makes full use of the relationships between all samples in mini-batch sampling.

2. The EEG signal recognition method based on feature embedding enhancement and coupling as described in claim 1, characterized in that, Step 1: EEG signals were acquired using a Brain Products 32-lead EEG cap, collecting four types of motor imagery signals from the subject: images of moving the left hand, right hand, both feet, and tongue; the signals were filtered using a Butterworth filter in the range of 0.5Hz-40Hz to extract information related to motor imagery, and then normalized using z-scores.

3. The EEG signal recognition method based on feature embedding enhancement and coupling as described in claim 1, characterized in that, Step 2 involves data augmentation of the preprocessed EEG signals, using random methods such as data flipping, channel swapping, and pruning. Specifically, this includes: Data augmentation involves randomly selecting a certain number of samples from the entire dataset in each training iteration and applying random operations to augment the original data. For example, a sample may be flipped or have its channels swapped.

4. The EEG signal recognition method based on feature embedding enhancement and coupling as described in claim 1, characterized in that, Step 4 involves concatenating the data from step 2 with the original data. Training a feature-enhanced coupled neural network specifically includes: When training the feature-enhanced coupled neural network, multiple loss functions are used for gradient updates during model training; the task loss function is the cross-entropy function, and its formula is: loss cls =-∑p i logq i In the formula, p represents the expected value and q represents the predicted value; The feature space was spatially constrained by two methods: difference loss and self-similarity loss. The formula for difference loss is as follows: In the formula, ||.|| represents the Frobenius norm; The formula for self-similarity loss is as follows: In the formula, mix(·) represents minimization, N is the number of samples, and E is an identity matrix.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the EEG signal recognition method based on feature embedding enhancement and coupling as described in any one of claims 1 to 4.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the EEG signal recognition method based on feature embedding enhancement and coupling as described in any one of claims 1 to 4.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the EEG signal recognition method based on feature embedding enhancement and coupling as described in any one of claims 1 to 4.

Images