A method, system and equipment for steady-state visual evoked potential analysis

Abstract

This invention provides a method, system, and device for steady-state visual evoked potential (SSVEP) analysis, relating to the field of SSVEP analysis. The method includes: acquiring multi-channel EEG signals and preprocessing the multi-channel EEG signals to generate SSVEP signals; extracting time-domain features of the SSVEP signals using a time-domain feature extraction module; extracting frequency-domain features of the SSVEP signals using a frequency-domain feature extraction module; fusing the time-domain features and the frequency-domain features to generate time-frequency fusion features; passing the time-frequency fusion features through an attention mechanism module to generate a weighted loss function represented by a set of feature weights and deep features; inputting the deep features into a binary classifier based on the weighted loss function, and outputting the SSVEP classification result; the number of binary classifiers is equal to the dimension of the deep features. This invention can improve the accuracy of SSVEP classification results.

Description

Technical Field

[0001] This invention relates to the field of SSVEP analysis, and in particular to a method, system and device for steady-state visual evoked potential analysis. Background Technology

[0002] Most high-performance models for identifying steady-state visual evoked potential (SSVEP) frequencies are linear. However, SSVEPs collected from different channels may have non-linear relationships with each other. Linear combinations of EEG signals from multiple channels are not the most accurate solution for SSVEP classification.

[0003] To obtain multi-dimensional electroencephalogram (EEG) signals, time-frequency analysis methods are used to perform preliminary feature extraction. However, due to the limited features and information extracted by a single network structure, it is impossible to fully explore the time-frequency information in the signal, making it difficult to guarantee accuracy and resulting in low accuracy of SSVEP classification results. Summary of the Invention

[0004] The purpose of this invention is to provide a steady-state visual evoked potential analysis method, system, and device to solve the problem of low accuracy in SSVEP classification results.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for analyzing steady-state visual evoked potentials includes:

[0007] Acquire multi-channel EEG signals and preprocess the multi-channel EEG signals to generate SSVEP signals;

[0008] The time-domain features of the SSVEP signal are extracted using the time-domain feature extraction module;

[0009] The frequency domain features of the SSVEP signal are extracted using the frequency domain feature extraction module.

[0010] By fusing the time-domain features and the frequency-domain features, a time-frequency fusion feature is generated.

[0011] The time-frequency fusion features are processed through an attention mechanism module to generate a weighted loss function and deep features represented by a set of feature weights;

[0012] Based on the weighted loss function, the deep features are input into a binary classifier, and the SSVEP classification result is output; the number of binary classifiers is equal to the dimension of the deep features.

[0013] Optionally, the preprocessing of the multi-channel EEG signals to generate SSVEP signals specifically includes:

[0014] The multi-channel EEG signals are filtered to generate filtered EEG signals;

[0015] The filtered EEG signal is subjected to Discrete Fourier Transform to generate an SSVEP signal.

[0016] Optionally, the time-domain feature extraction module comprises a three-layer convolutional neural network and a 75% random deactivation layer connected to the convolutional neural network. Within each time window of the convolutional neural network, convolution operations are performed on the multi-channel EEG signals to convert them into single-channel SSVEP signals. The first layer of the convolutional neural network has 16 9×9 convolutional kernels, the second layer has one 1×9 convolutional kernel, and the third layer has one 1×9 convolutional kernel. The convolutional neural network is used to mine the time-domain and inter-channel information of the SSVEP signals and extract time-domain features. The random deactivation layer is used to regularize the convolutional neural network to prevent overfitting.

[0017] Optionally, the time-domain feature extraction module comprises a three-layer convolutional neural network and a 15% random deactivation layer connected to the convolutional neural network. Within each time window of the convolutional neural network, convolution operations are performed on the multi-channel EEG signals to convert them into frequency features of a single channel. The first layer of the convolutional neural network has 16 9×9 convolutional kernels, the second layer has one 1×9 convolutional kernel, and the third layer has one 1×9 convolutional kernel. The convolutional neural network is used to mine the frequency domain and inter-channel information of the SSVEP signal and extract frequency domain features. The random deactivation layer is used to regularize the convolutional neural network to prevent overfitting.

[0018] Optionally, the attention mechanism module specifically includes: two fully connected layers and a multi-head classifier connected to the last fully connected layer; a ReLU activation function and a batch normalization layer are inserted between the two fully connected layers; a Tanh activation function is added after the last fully connected layer; the multi-head classifier is used to obtain the attention weights of all dimensions of the time-frequency fusion feature.

[0019] Optionally, the deep feature e is:

[0020]

[0021] Among them, F fusionFor time-frequency fusion features, W1 is the weight of the first fully connected layer, W2 is the weight of the first fully connected layer, b1 is the bias of the first fully connected layer, b2 is the bias of the second fully connected layer, BN is batch normalization, relu is the ReLU activation function, tanh is the Tanh activation function; T is the transpose matrix.

[0022] Optionally, the weighted loss function L SC for:

[0023]

[0024] Where k is the total number of samples, n is the feature dimension of the samples, and x mn Let y be an n-dimensional feature of the m-th sample. m Let Q be the category to which the m-th sample belongs. mn Let n be the feature weights of the m-th sample. Let ym be the nth feature center, r be the total dimension of the sample features, and ⊙ be the Hadamard product.

[0025] A steady-state visual evoked potential analysis system, comprising:

[0026] The SSVEP signal generation module is used to acquire multi-channel EEG signals and preprocess the multi-channel EEG signals to generate SSVEP signals.

[0027] A time-domain feature extraction module is used to extract the time-domain features of the SSVEP signal according to the time-domain feature extraction module.

[0028] A frequency domain feature extraction module is used to extract the frequency domain features of the SSVEP signal according to the frequency domain feature extraction module.

[0029] A time-frequency fusion feature generation module is used to fuse the time-domain features and the frequency-domain features to generate time-frequency fusion features;

[0030] The weighted loss function and deep feature generation module are used to generate a weighted loss function and deep features represented by a set of feature weights by passing the time-frequency fusion features through the attention mechanism module.

[0031] The SSVEP classification result output module is used to input the deep features into a binary classifier based on the weighted loss function and output the SSVEP classification result; the number of binary classifiers is equal to the dimension of the deep features.

[0032] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor running the computer program to enable the electronic device to perform the above-described steady-state visual evoked potential analysis method.

[0033] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described steady-state visual evoked potential analysis method.

[0034] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: The present invention provides a steady-state visual evoked potential analysis method, system and device, which extracts the time domain and frequency domain features of the EEG signal and fuses them, and performs recognition based on the attention mechanism, thereby improving the classification accuracy. Even on subjects with poor SSVEP performance, a good classification effect is achieved, further improving the performance of the SSVEP-based brain-computer interface (BCI). Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 The flowchart of the steady-state visual evoked potential analysis method provided by the present invention is shown below;

[0037] Figure 2 A simplified flowchart of the steady-state visual evoked potential analysis method provided by this invention;

[0038] Figure 3 This is a schematic diagram of the time-domain feature extraction module provided by the present invention;

[0039] Figure 4 This is a schematic diagram of the frequency domain feature extraction module provided by the present invention;

[0040] Figure 5 A schematic diagram of the attention mechanism module provided by the present invention;

[0041] Figure 6 This diagram illustrates a comparison of the average accuracy and information transfer rate (ITR) of the T-FNET method and ResNet18 across all subjects within different time windows, as provided in this invention. Figure 6 (a) in the figure is the average accuracy graph; Figure 6 (b) in the figure is the Information Transfer Rate (ITR) graph. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] The purpose of this invention is to provide a steady-state visual evoked potential analysis method, system, and device that can improve the accuracy of SSVEP classification results.

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] Example 1

[0046] like Figures 1-2 As shown, the present invention provides a method for analyzing steady-state visual evoked potentials, comprising:

[0047] Step 101: Acquire multi-channel EEG signals and preprocess the multi-channel EEG signals to generate SSVEP signals.

[0048] In practical applications, step 101 specifically includes: filtering the multi-channel EEG signal to generate a filtered EEG signal; and performing a discrete Fourier transform on the filtered EEG signal to generate an SSVEP signal.

[0049] Furthermore, in order to better conduct data analysis, the experimental data must first be preprocessed.

[0050] This invention was validated on the SSVEP benchmark dataset. All EEG data were filtered using a Chebyshev Type I filter with a cutoff frequency of 6Hz–90Hz and a stopband angular frequency of 4Hz–100Hz. A Fast Fourier Transform (FFT) was performed on the filtered EEG data for frequency-domain feature extraction. Since the step size of each SSVEP stimulus frequency was 0.2Hz, the frequency resolution of the FFT was also set to 0.2Hz. Data in the 5Hz–55Hz range was retained from each transformed frequency domain segment.

[0051] Since each EEG signal test includes 0.5 seconds before and 0.5 seconds after stimulation, this invention uses only 0.5 to 5.5 seconds of data for frequency identification. Nine channels are used: Pz, PO5, PO3, POz, PO4, PO6, O1, Oz, and O2.

[0052] Step 102: Extract the time domain features of the SSVEP signal using the time domain feature extraction module.

[0053] In practical applications, the time-domain feature extraction module consists of a three-layer convolutional neural network and a 75% random deactivation layer connected to the convolutional neural network. Within each time window of the convolutional neural network, convolution operations are performed on the multi-channel EEG signals to convert them into single-channel SSVEP signals. The first layer of the convolutional neural network has 16 9×9 convolutional kernels, the second layer has one 1×9 convolutional kernel, and the third layer has one 1×9 convolutional kernel. The convolutional neural network is used to mine the temporal and inter-channel information of the SSVEP signals and extract temporal features. The random deactivation layer is used to regularize the convolutional neural network to prevent overfitting.

[0054] Furthermore, such as Figure 3 As shown, to obtain the temporal features of the SSVEP signal, a three-layer convolutional neural network (CNN) is used to extract the temporal features of the EEG signal, defined as X:

[0055]

[0056] This three-layer CNN is named T-CNN. T-CNN performs convolution operations on multi-channel EEG signals within each small time window, converting the collected EEG signals from multiple channels into signals from a single channel. For the extracted one-dimensional time-domain signal, N t N represents the time duration. c Number of channels, N t =50, 100, 150, 200, and 250, corresponding to time windows of 0.2s, 0.4s, 0.6s, 0.8s, and 1.0s (50 is used as an example here), N cTo determine the number of channels, T-CNN has 16 9x9 convolutional kernels in the first layer, one 1x9 kernel in the second layer, and one 1x9 kernel in the third layer. The input data format is 50x9x1. Because of the use of same padding, the data scale remains unchanged. After each convolutional layer, the data formats are 50x9x16 and 50x9x1, respectively. The final output data format is 50x1x1. T-CNN convolves the EEG signals in all input channels to mine the temporal and inter-channel information of the SSVEP signal. It extracts temporal features as input for the subsequent correlation analysis module. After the three convolutional layers, a 75% random deactivation layer (Dropout) is added to regularize the network model to prevent overfitting.

[0057] Step 103: Extract the frequency domain features of the SSVEP signal using the frequency domain feature extraction module.

[0058] In practical applications, the time-domain feature extraction module consists of a three-layer convolutional neural network and a 15% random deactivation layer connected to the convolutional neural network. Within each time window of the convolutional neural network, convolution operations are performed on the multi-channel EEG signals to convert them into frequency features of a single channel. The first layer of the convolutional neural network has 16 9×9 convolutional kernels, the second layer has one 1×9 convolutional kernel, and the third layer has one 1×9 convolutional kernel. The convolutional neural network is used to mine the frequency domain and inter-channel information of the SSVEP signal and extract frequency domain features. The random deactivation layer is used to regularize the convolutional neural network to prevent overfitting.

[0059] Furthermore, such as Figure 4 As shown, to obtain the frequency domain features of the SSVEP signal, a three-layer convolutional neural network with a structure similar to the time domain feature extraction module is used to extract the frequency domain features of the EEG signal, defined as Y:

[0060]

[0061] This two-layer CNN is named F-CNN. It convolves EEG signals from multiple channels and outputs the frequency features of a single channel. For the extracted one-dimensional frequency domain signal, N t N represents the time duration. c N represents the number of channels. fCorresponding to 40 character categories that flash at different frequencies, the first layer has 40 9*1 convolutional kernels, and the second layer has one 1*40 convolutional kernel. The input data format is 50*40*9, and the data format after each convolutional layer is 50*40*40. The final output data format is 50*1*1. F-CNN mines the frequency domain and inter-channel information of the SSVEP signal, extracts frequency domain features, and uses them as input for the subsequent correlation analysis module. After the two convolutional layers, a 15% random deactivation layer is added to regularize the network model to prevent overfitting.

[0062] Step 104: Fuse the time domain features and the frequency domain features to generate time-frequency fusion features.

[0063] Furthermore, to address the issue that a single feature of the SSVEP signal is insufficient to fully represent the characteristics of the SSVEP signal, a recognition method based on time-frequency domain fusion is proposed. This method extracts and combines time and frequency domain features to construct a new feature vector that contains more information than the original multi-dimensional feature signal. The time-domain and frequency-domain feature vectors extracted by the pre-feature extraction module are then concatenated into a joint time-frequency fusion feature vector. The calculation method is as follows:

[0064]

[0065] Among them, F fusion For time-frequency fusion characteristics, and Using matrix multiplication, the time-domain features and frequency-domain features are calculated into a 50x50 feature fusion matrix. yes Transpose of a matrix.

[0066] Step 105: The time-frequency fusion features are processed through the attention mechanism module to generate a weighted loss function and deep features represented by a set of feature weights.

[0067] In practical applications, the weighted loss function L SC for:

[0068]

[0069] Where k is the total number of samples, n is the feature dimension of the samples, and x mn Let y be an n-dimensional feature of the m-th sample. m Let Q be the category to which the m-th sample belongs. mn Let n be the feature weights of the m-th sample. For y m The nth feature center, r is the total dimension of the sample features, and ⊙ is the Hadamard product.

[0070] In practical applications, such as Figure 5 As shown, the attention mechanism module specifically includes: two fully connected layers and a multi-head classifier connected to the last fully connected layer; a ReLU activation function and a batch normalization layer are inserted between the two fully connected layers; a Tanh activation function is added after the last fully connected layer; the multi-head classifier is used to obtain the attention weights of all dimensions of the time-frequency fusion feature.

[0071] In practical applications, the deep feature e is:

[0072]

[0073] Among them, F fusion For time-frequency fusion features, W1 is the weight of the first fully connected layer, W2 is the weight of the first fully connected layer, b1 is the bias of the first fully connected layer, b2 is the bias of the second fully connected layer, BN is batch normalization, relu is the ReLU activation function, tanh is the Tanh activation function; T is the transpose matrix.

[0074] Furthermore, to highlight the importance of features at different nodes and frequencies in the time series, an attention mechanism module was constructed. This module contains two trainable fully connected layers to obtain the specific correlation information between the respective feature dimensions. A ReLU activation function and a batch normalization layer are inserted between the fully connected layers to obtain the non-linear information between each layer. A Tanh activation function is added after the two fully connected layers to address gradient vanishing and preserve positive and negative activation values. Finally, a multi-head classifier is used to obtain the attention weights for all dimensions of the features.

[0075] The deep features *e* output by the attention mechanism module are passed to subsequent binary classifiers. The number of classifiers is equal to the dimension of the deep features. *e* is shared among all classifiers. The output consists of two parameters obtained after importance evaluation for each dimension of the deep features: 1. Relevance score, 2. Irrelevant score, which can be represented as follows:

[0076]

[0077]

[0078] Where we is the superscript of the relevance score, ex is the superscript of the irrelevance score, m is the number of samples, and n is the number of features. and Let the bias and weights be the values ​​used to calculate the relevance score for the nth classifier. and These are the biases and weights for calculating irrelevant scores for the nth classifier. For the relevant scores, As irrelevant scores, each feature vector can ultimately be represented as follows:

[0079]

[0080] The network backbone uses ResNet-18. The network's loss function combines a weighted center loss function and a softmax loss function to extract class-specific features, reduce the differences between classifications, and improve the network's generalization ability. Finally, the data passes through a fully connected layer for classification. The input data format is 50*50*1, and the network's output data format is 512*1*1.

[0081] Step 106: Based on the weighted loss function, input the deep features into a binary classifier and output the SSVEP classification result; the number of binary classifiers is equal to the dimension of the deep features.

[0082] This study proposes a novel SSVEP classification model, T-FNET, which classifies multi-target SSVEPs using data from individual subjects. A comparative experiment was conducted to compare the proposed SSVEP analysis method based on a time-frequency domain fusion network with commonly used basic TRCA and ensemble TRCA methods in the SSVEP recognition field. The method extracts time-domain and frequency-domain features from SSVEP signals and fuses these features, preserving the information contained in the original time-frequency domain signals. An attention mechanism module is constructed to obtain the weights of each feature, and a weighted center loss function is introduced to jointly guide network training, reducing intra-class discrepancies, enhancing feature discriminative power, and effectively improving classification accuracy. The T-FNET model was tested using a benchmark dataset of 35 subjects and 40 SSVEP frequencies. Table 1 shows the average accuracy and ITR of the two TRCA methods and T-FNET for all subjects in different time windows.

[0083] Table 1

[0084]

[0085] The SSVEP benchmark dataset was used to conduct six experiments on subjects. Data from one experiment was used as the test dataset, and the other five were used as the training dataset. This process was repeated six times to test all data. Paired-samples t-tests were performed to verify the statistical differences between T-FNET and other methods. The results show that compared to basic TRCA and ensemble TRCA, this model has significantly higher classification accuracy and Information Translate Rate (ITR). T-FNET, which combines convolutional neural networks and time-frequency feature fusion, outperforms existing methods.

[0086] This invention conducted ablation experiments on features weighted by an attention mechanism and features without an attention mechanism, such as... Figure 6 (a) and Figure 6 As shown in (b) of the paper, it can be seen that the present invention has advantages in improving recognition accuracy and ITR.

[0087] In summary, this invention provides T-FNET, a nonlinear convolutional neural network model based on time-frequency feature fusion. It converts multi-channel EEG signals into single-channel signals to combine EEG signals from different channels and times, combining the time-domain and frequency-domain information of the signals to construct fused features with mixed information. The fusion and complementarity of the two modes improve the correlation of the features. The attention mechanism module uses a squeeze-excitation architecture to obtain information weight scores for each dimension of the fused features, further enhancing the discriminative power of the feature information. Validation classification on the SSVEP benchmark dataset demonstrates that T-FNET has superior performance compared to other methods. The CNN network used in the feature extraction module effectively prevents overfitting. The overall convolutional neural network structure is easy to explore and adjust for better performance, effectively utilizing the advantages of neural networks and flexibility. This provides a reference for applying this architecture based on convolutional neural networks, combining attention mechanisms and feature fusion, to the classification of various signals.

[0088] Example 2

[0089] In order to perform the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a steady-state visual evoked potential analysis system is provided below.

[0090] A steady-state visual evoked potential analysis system, comprising:

[0091] The SSVEP signal generation module is used to acquire multi-channel EEG signals and preprocess the multi-channel EEG signals to generate SSVEP signals.

[0092] The time-domain feature extraction module is used to extract the time-domain features of the SSVEP signal according to the time-domain feature extraction module.

[0093] The frequency domain feature extraction module is used to extract the frequency domain features of the SSVEP signal according to the frequency domain feature extraction module.

[0094] The time-frequency fusion feature generation module is used to fuse the time domain features and the frequency domain features to generate time-frequency fusion features.

[0095] The weighted loss function and deep feature generation module are used to generate a weighted loss function and deep features represented by a set of feature weights by passing the time-frequency fusion features through the attention mechanism module.

[0096] The SSVEP classification result output module is used to input the deep features into a binary classifier based on the weighted loss function and output the SSVEP classification result; the number of binary classifiers is equal to the dimension of the deep features.

[0097] Example 3

[0098] This invention provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the steady-state visual evoked potential analysis method provided in Embodiment 1.

[0099] In practical applications, the aforementioned electronic devices can be servers.

[0100] In practical applications, electronic devices include: at least one processor, memory, bus, and communication interface.

[0101] The processor, communication interface, and memory communicate with each other via a communication bus.

[0102] A communication interface is used to communicate with other devices.

[0103] The processor is used to execute programs, specifically the methods described in the above embodiments.

[0104] Specifically, the program may include program code, which includes computer operation instructions.

[0105] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The electronic device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0106] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive.

[0107] Based on the description of the above embodiments, this application provides a storage medium storing computer program instructions thereon, which can be executed by a processor to implement the methods described in any embodiment.

[0108] The steady-state visual evoked potential analysis system provided in this application exists in various forms, including but not limited to:

[0109] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0110] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access capabilities. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0111] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0112] (4) Other electronic devices with data interaction functions.

[0113] Specific embodiments of the subject matter have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0114] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0115] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components. Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0120] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0121] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, and CD-ROM.

[0122] Digital multifunction optical disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape, disk storage or other magnetic storage devices

[0123] Or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0125] This application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific transactions or implement specific abstract data types. This application can also be practiced in distributed computing environments where transactions are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0126] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0127] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A steady-state visual evoked potential analysis method, characterized by, Comprise: acquire multi-channel electroencephalogram signals, and preprocess the multi-channel electroencephalogram signals to generate SSVEP signals; extract time domain features of the SSVEP signals according to a time domain feature extraction module; extract frequency domain features of the SSVEP signals according to a frequency domain feature extraction module; fuse the time domain features and the frequency domain features to generate time-frequency fusion features; The time-frequency fusion features are input into an attention mechanism module to generate a weighted loss function represented by a set of feature weights and deep features. are: wherein ， is a time-frequency fusion feature, is a weight of the first fully connected layer, is a weight of the first fully connected layer, is a bias of the first fully connected layer, is a bias of the second fully connected layer BN is a batch normalization process , relu is a ReLU activation function, tanh is a Tanh activation function; T is a transpose matrix; based on the weighted loss function, input the deep features into a binary classifier to output SSVEP classification results; the number of the binary classifier is equal to the dimension of the deep features.

2. The steady-state visually evoked potential analysis method according to claim 1, characterized by, The preprocessing of the multi-channel electroencephalogram signals to generate SSVEP signals specifically comprises: filtering the multi-channel electroencephalogram signals to generate filtered electroencephalogram signals; performing discrete Fourier transform on the filtered electroencephalogram signals to generate SSVEP signals.

3. The steady-state visually evoked potential analysis method according to claim 1, characterized by, The time domain feature extraction module is a three-layer convolutional neural network and a 75% random inactivation layer connected with the convolutional neural network; the multi-channel electroencephalogram signals are provided with convolution operation in each time window in the convolutional neural network to convert the multi-channel electroencephalogram signals into single-channel SSVEP signals; the first layer of the convolutional neural network has 16 9x9 convolution kernels, the second layer of the convolutional neural network has a 1x9 convolution kernel, and the third layer of the convolutional neural network has a 1x9 convolution kernel; the convolutional neural network is used to mine the information of the time domain and the inter-channel of the SSVEP signals to extract time domain features; and the random inactivation layer is used to regularize the convolutional neural network to prevent overfitting.

4. The steady-state visually evoked potential analysis method according to claim 1, characterized by, The frequency domain feature extraction module is a three-layer convolutional neural network and a 15% random inactivation layer connected with the convolutional neural network; the multi-channel electroencephalogram signals are provided with convolution operation in each time window in the convolutional neural network to convert the multi-channel electroencephalogram signals into single-channel frequency features; the first layer of the convolutional neural network has 16 9x9 convolution kernels, the second layer of the convolutional neural network has a 1x9 convolution kernel, and the third layer of the convolutional neural network has a 1x9 convolution kernel; the convolutional neural network is used to mine the information of the frequency domain and the inter-channel of the SSVEP signals to extract frequency domain features; and the random inactivation layer is used to regularize the convolutional neural network to prevent overfitting.

5. The steady-state visually evoked potential analysis method according to claim 1, wherein, The attention mechanism module specifically comprises: two fully connected layers and a multi-head classifier connected with the last fully connected layer; a ReLU activation function and a batch normalization layer are inserted between the two fully connected layers; a Tanh activation function is added after the last fully connected layer; and the multi-head classifier is used to obtain attention weights of all dimensions of the time-frequency fusion features.

6. The steady-state visually evoked potential analysis method according to claim 1, wherein, The weighted loss function is: where k is the total number of samples, n is the feature dimension of the samples, is the n-dimensional feature of the mth sample, is the class to which the mth sample belongs, is the n-dimensional feature weight of the mth sample, is the n-dimensional feature center of the mth sample, is the total dimension of the sample features, and is the Hadamard product.

7. A steady-state visual evoked potential analysis system, characterized in that, Comprise: an SSVEP signal generation module configured to acquire multi-channel electroencephalogram signals, and preprocess the multi-channel electroencephalogram signals to generate SSVEP signals; a time domain feature extraction module configured to extract time domain features of the SSVEP signal according to the time domain feature extraction module; a frequency domain feature extraction module configured to extract frequency domain features of the SSVEP signal according to the frequency domain feature extraction module; a time-frequency fusion feature generation module configured to fuse the time domain features and the frequency domain features to generate time-frequency fusion features; The weighted loss function and the deep feature generation module are configured to generate a weighted loss function represented by a set of feature weights and a deep feature by passing the time-frequency fusion feature through an attention mechanism module. is: wherein ， is a time-frequency fusion feature, is a weight of the first fully connected layer, is a weight of the first fully connected layer, is a bias of the first fully connected layer, is a bias of the second fully connected layer BN is a batch normalization process relu is a ReLU activation function, tanh is a Tanh activation function; T is a transpose matrix; an SSVEP classification result output module configured to input the deep features into a binary classifier based on the weighted loss function to output SSVEP classification results; the number of the binary classifiers is equal to the dimension of the deep features.

8. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to make the electronic device execute the steady-state visual evoked potential analysis method according to any one of claims 1-6.

9. A computer readable storage medium, characterized in that, The computer program is stored in the memory and is executed by the processor to realize the steady-state visual evoked potential analysis method according to any one of claims 1-6.

Images