SSVEP classification method based on convolutional neural network

Abstract

The invention provides an SSVEP classification method based on a convolutional neural network, and relates to the technical field of data processing, and the method comprises the steps: S1, obtaining an original SSVEP signal, and carrying out the ensemble empirical mode decomposition of the SSVEP signal into an intrinsic mode function through an ICEEMDAN algorithm; s2, inputting an intrinsic mode function obtained by decomposition into a time feature extractor based on a graph convolutional network to perform time feature data extraction; s3, inputting the time feature data extracted in the step S2 into a spatial feature extractor based on a graph convolutional network for spatial feature data extraction; s4, performing feature fusion on the data after feature extraction through a compression excitation function, and performing classification according to features of the fused data to obtain a classification result; according to the method, the classification accuracy and the information transmission rate can be remarkably improved, and the influence of noise and artifacts is reduced.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to an SSVEP classification method based on convolutional neural networks. Background Technology

[0002] The development of brain-computer interface (BCI) technology has provided a way to directly control external devices through brain signals. As a non-invasive BCI technology, EEG-based technology interprets and identifies EEG signals using algorithms, ultimately converting them into usable commands. Among various EEG-based BCI technologies, steady-state visual evoked potentials (SSVEPs) have received widespread attention and research due to their advantages such as high information transmission rate and low user training requirements. SSVEPs are generated when a user focuses on stimuli with different flashing frequencies; the visual cortex of the brain oscillates accordingly, resulting in stronger amplitudes of the corresponding frequencies and harmonics in the EEG signal.

[0003] Current SSVEP classification methods can be broadly categorized into traditional methods and deep learning-based methods. Traditional methods, such as Power Spectral Density Analysis (PSDA), Canonical Correlation Analysis (CCA), and Filter Bank Canonical Correlation Analysis (FBCCA), are mostly based on fixed mathematical models, obtaining classification results by matching or analyzing the correlation between EEG signals and stimulus targets one by one. Deep learning-based SSVEP classification methods utilize time-domain or frequency-domain EEG signals as input and employ convolutional neural networks for automatic feature extraction and learning, ultimately leading to classification. Because convolutional neural networks can automatically adjust network weight parameters based on features contained in the training data, they exhibit better fitting characteristics compared to traditional algorithms based on fixed mathematical models. However, current deep learning-based methods rely excessively on calibration data from individual subjects, failing to effectively reduce the influence of noise and artifacts, thus hindering further improvements in classification accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide an SSVEP classification method based on convolutional neural networks, which can significantly improve classification accuracy and information transmission rate, and reduce the impact of noise and artifacts.

[0005] The technical solution of this invention is as follows:

[0006] In a first aspect, this application provides an SSVEP classification method based on a convolutional neural network, which includes the following steps:

[0007] S1. Obtain the original SSVEP signal and use the ICEEMDAN algorithm to perform ensemble empirical mode decomposition of the SSVEP signal into intrinsic mode functions.

[0008] S2. Input the intrinsic mode functions obtained from the decomposition into a time feature extractor based on a graph convolutional network to extract time feature data;

[0009] S3. Input the temporal feature data extracted in step S2 into a spatial feature extractor based on a graph convolutional network to extract spatial feature data.

[0010] S4. The extracted data is fused using a compression activation function, and the data is classified based on the features of the fused data to obtain the classification result.

[0011] Further, in step S1, the above-mentioned calculation process of using the ICEEMDAN algorithm to perform ensemble empirical mode decomposition of the SSVEP signal into eigenmode functions includes:

[0012] Let x be a channel of the SSVEP signal to be decomposed, and add k sets of Gaussian white noise to channel x:

[0013] x k =x+β1Ε1(ω k )

[0014] This leads to the first IMF:

[0015]

[0016] Thus, the first residual R1 is obtained:

[0017] R1 = x - C1

[0018] After continuously adding white noise to R1, we obtain the second residual R2 and the second IMFC2:

[0019]

[0020] Infer the nth residual R n and intrinsic mode functions C n :

[0021]

[0022] C n =R n-1 -R n

[0023] In the formula, x k Let x be the sequence to be decomposed, k be the k-th group of Gaussian white noise, and β1, β2, β3, β4, β5, β6, β7, β8, β9, β1 ... n All are constants, E1, E2, E n Both represent the intrinsic mode function operators obtained after mode decomposition calculation. is x kThe first eigenmode function, C1 is the first eigenmode function of the input signal x, R1 represents the first residual, R2 represents the second residual, M is the local operator, K is the total amount of Gaussian white noise used in empirical mode decomposition, ω k Add the k-th group to the residual signal R n The Gaussian white noise sequence is given, where n represents the number of subbands and R0 is the sum of the values. n Let C represent the nth residual. n This represents the nth intrinsic mode function.

[0024] Furthermore, step S2 also includes performing BatchNorm processing on the aforementioned time feature extractor and introducing the PReLU activation function for processing.

[0025] Furthermore, in step S3, the calculation process for spatial feature data extraction by the aforementioned spatial feature extractor includes:

[0026]

[0027] L=DA

[0028] L=UΛU T

[0029]

[0030] In the formula, A is the adjacency matrix, and dist is the adjacency matrix. (i,j) Let $\frac{\pi}{\mathbf{a}$ be the distance between two nodes, $i$ and $j$ represent the row and column of the node, respectively, $θ$ and $γ$ are fixed parameters, $L$ is the Laplacian matrix, $D$ is the diagonal matrix, $U$ is the eigenvalue matrix, $Λ$ is the diagonal matrix, $x$ is the SSVEP signal to be decomposed, and $y$ is a set of data in the graph signal. Let x be the projection of x onto the eigenvector basis of the graph Laplacian matrix. Represents convolution, ⊙ represents element-wise multiplication, g θ For the filter, K represents the Chebyshev polynomial parameters. T is the maximum value among the diagonal elements. K (x) is a Chebyshev polynomial. For the normalized graph Laplace matrix, g′ θ This is a graph convolution filter.

[0031] Furthermore, in step S4, the process of feature fusion of the extracted data using a compressed activation function includes:

[0032]

[0033] s = F ex (z,W)=σ(W2δ(W1z))

[0034]

[0035] In the formula, z c Let U be the feature obtained from the c-th compression, and U be the input signal. c For the c-th channel of input U, F sq For compression functions, H and W are parameters, and F... ex Let represent the extraction function, σ be the activation function, W1 represent the dimensionality reduction layer, W2 represent the dimensionality increase layer, s be the output of the activation operation, and z be the compressed features. For the final fusion result, F scale A function that multiplies channels element by element.

[0036] Furthermore, step S4 also includes: evaluating the classification results based on classification accuracy and information transmission rate, calculated using the following formula:

[0037]

[0038] In the formula, ITR is the amount of information transmitted by the BCI system per unit time, usually expressed in bits per minute, T is the average selection time, N is the total number of categories in the current dataset, and P is the recognition accuracy.

[0039] Secondly, this application provides an electronic device, comprising:

[0040] Memory, used to store one or more programs;

[0041] processor;

[0042] When one or more of the above programs are executed by the above processor, an SSVEP classification method based on a convolutional neural network, as described in any of the first aspects above, is implemented.

[0043] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an SSVEP classification method based on a convolutional neural network as described in any of the first aspects above.

[0044] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:

[0045] (1) The present invention provides an SSVEP classification method based on convolutional neural network, which decomposes non-stationary signal modes into a set of intrinsic mode functions (IMF) through the ICEEMDAN algorithm, thereby effectively utilizing different vibration mode information, reducing the influence of noise and artifacts, and improving classification accuracy;

[0046] (2) In this invention, each IMF is convolved on the time dimension to extract the temporal features. Then, the data is input into the spatial feature extractor of the initialized adjacency matrix. The dynamic graph convolutional neural network is used to learn the weights between the nodes and edges, so that the network can adaptively adjust the relationship between different nodes. Finally, the feature is fused by compressing the activation function, so that the convolutional neural network model can more adaptively learn the weights of each channel, thereby improving the classification accuracy and information transmission rate. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the steps of an SSVEP classification method based on a convolutional neural network according to the present invention.

[0049] Figure 2 This is a framework diagram of an SSVEP classification method based on a convolutional neural network according to the present invention;

[0050] Figure 3 This relates the spatial position between electrodes to the adjacency matrix.

[0051] Figure 4 This is a decomposition diagram of ICEEMDAN;

[0052] Figure 5 This is a schematic diagram of the experimental results for dataset A;

[0053] Figure 6 This is a schematic diagram of the experimental results for dataset B;

[0054] Figure 7 This is a schematic structural block diagram of an electronic device according to an embodiment of the present invention.

[0055] Icons: 101, memory; 102, processor; 103, communication interface. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0057] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0058] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0059] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the various embodiments and features described below can be combined with each other.

[0060] Example 1

[0061] See also Figure 1-2 , Figure 1 The diagram shows the steps of an SSVEP classification method based on a convolutional neural network provided in an embodiment of this application. Figure 2 The diagram shows the framework of an SSVEP classification method based on a convolutional neural network according to the present invention.

[0062] In a first aspect, this application provides an SSVEP classification method based on a convolutional neural network, which includes the following steps:

[0063] S1. Obtain the original SSVEP signal and use the ICEEMDAN algorithm to perform ensemble empirical mode decomposition of the SSVEP signal into intrinsic mode functions.

[0064] S2. Input the intrinsic mode functions obtained from the decomposition into a time feature extractor based on a graph convolutional network to extract time feature data;

[0065] S3. Input the temporal feature data extracted in step S2 into a spatial feature extractor based on a graph convolutional network to extract spatial feature data.

[0066] S4. The extracted data is fused using a compression activation function, and the data is classified based on the features of the fused data to obtain the classification result.

[0067] As a preferred implementation, step S1, which involves using the ICEEMDAN algorithm to perform ensemble empirical mode decomposition of the SSVEP signal into intrinsic mode functions, includes:

[0068] Let x be a channel of the SSVEP signal to be decomposed, and add k sets of Gaussian white noise to channel x:

[0069] xk =x+β1Ε1(ω k )

[0070] This leads to the first IMF:

[0071]

[0072] Thus, the first residual R1 is obtained:

[0073] R1 = x - C1

[0074] After continuously adding white noise to R1, we obtain the second residual R2 and the second IMFC2:

[0075]

[0076] Infer the nth residual R n and intrinsic mode functions C n :

[0077]

[0078] C n =R n-1 -R n

[0079] In the formula, x k Let x be the sequence to be decomposed, k be the k-th group of Gaussian white noise, and β1, β2, β3, β4, β5, β6, β7, β8, β9, β1 ... n All are constants, E1, E2, E n Both represent the intrinsic mode function operators obtained after mode decomposition calculation. is x k The first eigenmode function, C1 is the first eigenmode function of the input signal x, R1 represents the first residual, R2 represents the second residual, M is the local operator, K is the total amount of Gaussian white noise used in empirical mode decomposition, ω k Add the k-th group to the residual signal R n The Gaussian white noise sequence is given, where n represents the number of subbands and R0 is the sum of the values. n Let C represent the nth residual. n This represents the nth intrinsic mode function.

[0080] In a preferred embodiment, step S2 further includes performing BatchNorm processing on the time feature extractor and introducing the PReLU activation function for processing.

[0081] In a preferred embodiment, step S3, the calculation process for spatial feature data extraction by the spatial feature extractor includes:

[0082]

[0083] L=DA

[0084] L=UΛU T

[0085]

[0086]

[0087] In the formula, A is the adjacency matrix, and dist is the adjacency matrix. (i,j) Let $\frac{\pi}{\mathbf{a}$ be the distance between two nodes, $i$ and $j$ represent the row and column of the node, respectively, $θ$ and $γ$ are fixed parameters, $L$ is the Laplacian matrix, $D$ is the diagonal matrix, $U$ is the eigenvalue matrix, $Λ$ is the diagonal matrix, $x$ is the SSVEP signal to be decomposed, and $y$ is a set of data in the graph signal. Let x be the projection of x onto the eigenvector basis of the graph Laplacian matrix. Represents convolution, ⊙ represents element-wise multiplication, g θ For the filter, K represents the Chebyshev polynomial parameters. T is the maximum value among the diagonal elements. K (x) is a Chebyshev polynomial. For the normalized graph Laplace matrix, g′ θ This is a graph convolution filter.

[0088] The adjacency matrix is ​​typically used to describe the relationships between nodes in a graph convolution, recording their positional information, such as... Figure 3 The figure shows the relationship between the spatial position of the electrodes and the adjacency matrix. Figure (a) shows the distribution of brain channels, Figure (b) shows the collected data, Figure (c) shows the channel mapping matrix, and Figure (d) shows the matrix feature points.

[0089] In a preferred embodiment, step S4, the process of feature fusion of the feature-extracted data using a compression activation function, includes:

[0090]

[0091] s = F ex (z,W)=σ(W2δ(W1z))

[0092]

[0093] In the formula, z c Let U be the feature obtained from the c-th compression, and U be the input signal. c For the c-th channel of input U, F sq For compression functions, H and W are parameters, and F... exLet represent the extraction function, σ be the activation function, W1 represent the dimensionality reduction layer, W2 represent the dimensionality increase layer, s be the output of the activation operation, and z be the compressed features. For the final fusion result, F scale A function that multiplies channels element by element.

[0094] Example 2

[0095] This embodiment uses two publicly available SSVEP datasets for experiments:

[0096] The public dataset A consists of 12 target stimulus blocks arranged in a 4×3 sequence. Each target block is updated once per second on a 27-inch external display, with the target blocks designed for frequency (9.25Hz:0.5Hz:14.75Hz) and phase (0.0π:0.5π:1.5π). Researchers used the BioSemiActiveTwo EEG system to collect EEG data from 10 healthy subjects. The sampling frequency was 2048Hz, with 8 electrodes placed in the occipital region, specifically following the CMS electrode layout of the 10-20BCI system. Each subject completed 15 experimental blocks, with each block consisting of 12 target stimulus experiments. After the experiments, the EEG data for each subject was downsampled, reducing the sampling rate to 256Hz. The first 135 milliseconds of each experiment were used as visual delay data, while training data was acquired starting from the time point after the visual delay. The entire dataset contains 180 experiments.

[0097] The public dataset B consists of 40 target stimulus blocks arranged in a 5×8 sequence. Each target block is refreshed on the interface once per second, with the target blocks designed as (8Hz:0.2Hz:15.8Hz) and phase (0.0π:0.5π:1.5π). Researchers used the Synamps2 EEG system to collect EEG data from 35 healthy subjects. The data sampling frequency was 1000Hz, and the 64 channels were arranged using the international 10-20 electrode placement system. Each subject participated in 6 experimental blocks, each containing 40 targets, for a total of 240 experiments. After data acquisition, the EEG data for each subject was downsampled, reducing the sampling rate to 250Hz. In this study, nine electrodes—PO3, PO2, PO4, O1, O2, P2, PO5, and PO6—were selected as input channels.

[0098] This paper uses classification accuracy and information transfer rate (ITR) as performance metrics for the aforementioned methods. Classification accuracy is the ratio of the number of correctly classified samples to the total number of test samples. ITR reflects the amount of information transmitted by the BCI system per unit time, usually expressed in bits per minute, and its calculation formula is as follows:

[0099]

[0100] In the formula, ITR is the amount of information transmitted by the BCI system per unit time, usually expressed in bits per minute, T is the average selection time, N is the total number of categories in the current dataset, and P is the recognition accuracy.

[0101] In order to better simulate the actual recognition scenario, the experiment added a gaze target block time of 0.5s to the average selection time T.

[0102] This experiment is based on the Windows 10 operating system. Data preprocessing was performed using Matlab R2021b. Deep learning network-related operations were performed using PyTorch 1.13.0 under the PyCharm 2022 editor. The GPU used was an NVIDIA GeForce GTX 1660 SUPER (6GB RAM), and the CPU used was an AMD Ryzen 3700X 8-core processor.

[0103] For the two public datasets, this embodiment employs a leave-one-subject-out (LOSO) cross-validation strategy. Specifically, during model evaluation, the experiment uses data from one participant in the dataset as the test set and the data from the remaining participants as the training set. This process is repeated cyclically until every participant has been used as a test subject.

[0104] To evaluate the performance of the proposed method, this embodiment uses popular SSVEP methods such as TRCA, FB-EEGNet, C-CNN, and FB-CNN as baseline methods, and conducts experiments on public datasets A and B, respectively.

[0105] Regarding the selection of the number of ICEEMDAN subbands, based on the following results, the top three IMF functions were chosen. For the deep learning network, the convolutional kernel dimension in the temporal feature extractor is 1×10, the output channels are twice that of the input, and it is sequentially processed by the PReLU activation function and Dropout (0.5). For the spatial feature extractor, the maximum order was ultimately set to 3, and the output channels are the same as the input. For the feature fusion module, the input and output channels are the same, and the reduction ratio is set to 8. Fully connected classification is then performed. For the entire training process, the learning rate (lr) is set to 0.007, the batch size is set to 32, and the calculation parameters are updated using the Adam (beta1 = 0.9, beta2 = 0.999) optimizer.

[0106] To verify the effectiveness of the proposed method, this embodiment introduces machine learning methods and deep learning methods for comparison. The relevant descriptions of each baseline method are as follows:

[0107] TRCA: This method belongs to spatial filtering techniques. It learns a spatial filter by optimizing the repeatability among existing data, thereby extracting valuable components from the data and using the average value of the existing data as a reference signal. The spatial filter calculates the correlation coefficient between the projected features of the subject's test data and the reference signal, thus generating a classification result. In this experiment, the data of one subject was used as the test sample, and the data of the other subjects were used as calibration data.

[0108] FB-EEGNet: This method is a deep learning model proposed for EEG signals, and it has achieved excellent results in the SSVEP field. EEGNet mainly consists of four layers: convolutional layers that perform filtering operations on each channel, deep convolutional layers that weight the channels, separable convolutional layers for feature extraction, and fully connected layers. FB-EEGNet, on the other hand, uses EEGNet to extract features from multiple different harmonic signals separately.

[0109] C-CNN: This method first uses Fourier transform to convert the data from the time domain to the frequency domain, then uses the real and complex numbers of the transformed data to simulate frequency and phase information, which is then input into the network for feature extraction. The network structure is simple, consisting of two convolutional layers and one fully connected layer. The convolutional layers are responsible for learning the frequency and phase information, while the fully connected layer is responsible for summarizing the extracted features to obtain the classification result.

[0110] FB-CNN: Starting from the C-CNN classification method, researchers later proposed a filter bank-based C-CNN based on the concept of harmonics. This method uses a filter bank to decompose the original signal into multiple harmonics, which are then input into multiple branch networks for feature extraction. Finally, the features extracted from each branch are concatenated and summarized using a fully connected layer to obtain the classification result.

[0111] To better select the IMF after ICEEMDAN decomposition, participant 5 from the publicly available dataset A was used in the experiment, and the single channel of the Oz electrode in this participant was decomposed and visualized. Figure 4 It can be seen that the Oz channel is decomposed into 6 IMFs and 1 residual signal. According to the frequency domain information in the figure, the information contained in the IMFs is in an increasing relationship. The study sets up an accumulation experiment starting from IMF1. In 7 experiments accumulating a single IMF1 to all decomposed subbands, the accuracy rates within a 1.0s window were 85.51%, 89.22%, 93.06%, 91.33%, 90.27%, 87.14%, and 86.11%, respectively. The results show that the classification performance is optimal when selecting IMF1 to IMF3. Therefore, in this embodiment, feature extraction is performed on the first 3 IMF subbands using a deep learning network.

[0112] See also Figure 5 , Figure 5 In the figures (a) and (b), ICE-GSE and the baseline method respectively demonstrate the average classification accuracy and information transfer rate (ITR) obtained after experiments on public dataset A. The data length ranges from 0.5 seconds to 1.0 seconds, and the time step is 0.1 seconds. Among all time window lengths, the classification performance, from highest to lowest, is: ICE-GSE, FB-CNN, C-CNN, FB-EEGNet, and TRCA. The experimental results demonstrate that the proposed ICE-GSE outperforms other baseline methods on both public datasets. ICE-GSE achieves the best average classification accuracy and ITR at the selected time length. Within a time range of 0.5s to 1.0s, the average classification accuracy of ICE-GSE was 69.26%, 73.15%, 79.34%, 83.71%, 86.20%, and 89.36%, respectively, with ITRs of 97.88 bits / min, 99.10 bits / min, 106.77 bits / min, 109.86 bits / min, 108.37 bits / min, and 109.12 bits / min, respectively.

[0113] See also Figure 6 , Figure 6 In the figures (a) and (b), the average classification accuracy and ITR obtained by the above method on the public dataset B are respectively. The time window length spans from 0.5s to 1.0s, and the time step is 0.1s. The ICE-GSE network outperforms other baseline methods in both metrics, with average classification accuracies of 55.37%, 64.75%, 73.28%, 78.93%, 84.61%, and 85.22%, and ITRs of 118.28 bits / min, 137.59 bits / min, 153.61 bits / min, 159.95 bits / min, 166.67 bits / min, and 157.46 bits / min.

[0114] The results above show that, after comprehensive evaluation using two public datasets, ICE-GSE can be demonstrated to improve the classification performance of SSVEP in the between-subjects scenario.

[0115] Example 3

[0116] See also Figure 7 , Figure 7 This is a schematic structural block diagram of an electronic device provided in an embodiment of this application.

[0117] An electronic device includes a memory 101, a processor 102, and a communication interface 103. The memory 101, processor 102, and communication interface 103 are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules. The processor 102 executes the software programs and modules stored in the memory 101 to perform various functional applications and data processing. The communication interface 103 can be used for signaling or data communication with other node devices.

[0118] The memory 101 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0119] The processor 102 can be an integrated circuit chip with signal processing capabilities. The processor 102 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0120] It is understood that the structure shown in the figure is for illustrative purposes only. An SSVEP classification method based on convolutional neural networks may include more or fewer components than shown in the figure, or have a different configuration. The components shown in the figure can be implemented in hardware, software, or a combination thereof.

[0121] In the embodiments provided in this application, it should be understood that the disclosed methods can also be implemented in other ways. The embodiments described above are merely illustrative. For example, the flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0122] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0123] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0124] In summary, the SSVEP classification method based on convolutional neural networks provided in this application decomposes non-stationary signal modes into a set of intrinsic mode functions (IMFs) using the ICEEMDAN algorithm, thereby effectively utilizing information from different vibration modes and reducing the influence of noise and artifacts. Each IMF is then convolved with temporal information to extract temporal features. The data is then input into a spatial feature extractor with an initialized adjacency matrix. This graph convolutional network-based spatial feature extractor provides an effective method to describe the intrinsic relationships between different nodes in the EEG signal, and utilizes a dynamic graph convolutional neural network to learn the weights between node edges, enabling the network to adaptively adjust the relationships between different nodes. Finally, feature fusion is performed based on a compression excitation (SE) function, and channels relevant to the classification task are selected to achieve classification, significantly improving classification accuracy.

[0125] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0126] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application.

Claims

1. A SSVEP classification method based on convolutional neural network, characterized in that: The following steps are involved: S1. Obtain the original SSVEP signal and use the ICEEMDAN algorithm to perform collective empirical mode decomposition of the SSVEP signal into intrinsic mode functions; S2, inputting the decomposed intrinsic mode function into a time feature extractor based on a graph convolutional network to extract time feature data; S3, inputting the temporal feature data extracted in step S2 into a spatial feature extractor based on a graph convolutional network to extract spatial feature data; S4. Perform feature fusion on the feature-extracted data through a compression excitation function, and classify the data according to the features of the fused data to obtain a classification result.

2. The SSVEP classification method based on convolutional neural network according to claim 1, characterized in that: In step S1, the calculation process of using the ICEEMDAN algorithm to perform collective empirical mode decomposition of the SSVEP signal into intrinsic mode functions includes: x k =x+β1E1(ω k ) R1=x-C1 C n =R n-1 -R n In the formula, x k represents the sequence, x is the SSVEP signal to be decomposed, k represents the kth group of Gaussian white noise, β1, β2, β n are all constants, Ε1, Ε2, Ε n Both represent the intrinsic mode function operators obtained after modal decomposition calculation. For x k The first intrinsic mode function of the input signal x, C1 is the first intrinsic mode function of the input signal x, R1 is the first residual, R2 is the second residual, M is the local operator, K is the total number of Gaussian white noise used in the empirical mode decomposition, ω k Add the kth group to the residual signal R n The Gaussian white noise sequence in, n represents the number of decomposition subbands, R n represents the nth residual, C n represents the nth eigenmode function.

3. The SSVEP classification method based on convolutional neural network according to claim 1, characterized in that: Step S2 also includes performing batch normalization BatchNorm processing on the temporal feature extractor and introducing a PReLU activation function for processing.

4. The SSVEP classification method based on convolutional neural network according to claim 1, characterized in that: In step S3, the calculation process of the spatial feature extractor for extracting spatial feature data includes: L=DA L=UΛU T In the formula, A is the adjacency matrix, dist (i,j) is the distance between two nodes, i and j represent the rows and columns of the nodes respectively, θ and γ are both fixed parameters, L is the Laplace matrix, D is the diagonal matrix, U is the eigenvalue matrix, Λ is the diagonal matrix, x is the SSVEP signal to be decomposed, y is a set of data in the graph signal, is the projection of x on the eigenvector basis of the graph Laplacian matrix, represents convolution, ⊙ represents element-by-element multiplication, g θ is the filter, K represents the Chebyshev polynomial parameter, is the maximum value among the diagonal elements, T K (x) is a Chebyshev polynomial, is the normalized graph Laplacian matrix, g θ is a graph convolution filter.

5. The SSVEP classification method based on convolutional neural network according to claim 1, characterized in that: In step S4, the process of performing feature fusion on the feature-extracted data by using the compression excitation function includes: s=F ex (z,W)=σ(W2δ(W1z)) In the formula, z c is the feature obtained by the cth compression, U is the input signal, u c is the cth channel of input U, F sq is the compression function, H and W are parameters, F ex represents the extraction function, σ is the activation function, W1 represents the dimension reduction layer, W2 represents the dimension increase layer, s is the output of the excitation operation, z is the compressed feature, is the final fusion result, F scale A function that multiplies the channels element-wise.

6. The SSVEP classification method based on convolutional neural network according to claim 1, characterized in that: Step S4 also includes: evaluating the classification results based on the classification accuracy and information transmission rate, and the calculation formula is: Where ITR is the amount of information transmitted by the BCI system per unit time, usually in bits per minute, T is the average selection time, N is the total number of categories in the current data set, and P is the recognition accuracy.

7. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, a SSVEP classification method based on a convolutional neural network as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, an SSVEP classification method based on a convolutional neural network as described in any one of claims 1 to 6 is implemented.

Images