Visual evoked brain-computer interface system and method based on high-frequency time division multiplexing coding

Abstract

This invention provides a visually evoked brain-computer interface system and method based on high-frequency time-division multiplexing coding, relating to the field of brain-computer interfaces. The system includes: a visual stimulation module comprising multiple visual stimulation units; a high-frequency time-division multiplexing coding module pre-stored with multiple baseband symbols and coding rules; a signal acquisition and processing module located in the user's brain, which acquires the user's EEG signals after stimulation by the visual stimulation module and preprocesses the acquired EEG signals; a classification module connected to the signal acquisition and processing module, which predicts the coding pattern of the preprocessed EEG signals after passing through a classification model module; and an application control module with a pre-set instruction mapping table for converting the coding pattern output by the classification module into control instructions for external devices or software. The technical solution of this invention overcomes the problems of visual fatigue, excessive coding capacity, and low recognition accuracy caused by low-frequency coding in existing brain-computer interfaces.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interfaces, and more specifically to a visual evoked brain-computer interface system and method based on high-frequency time-division multiplexing coding. Background Technology

[0002] Steady-state visual evoked potentials (SSVEPs) are periodic electrophysiological responses of the visual cortex of the brain to visual stimuli of specific frequencies. The SSVEP-BCI brain-computer interface system presents flashing visual stimuli of different frequencies and identifies the corresponding frequency components in the EEG signals when the user gazes at a specific target, thereby interpreting the user's intentions and enabling human-computer interaction.

[0003] Currently, the traditional SSVEP-BCI system faces three core challenges: First, there are safety and comfort issues. Most mainstream systems use low-frequency encoding (e.g., 5-30Hz), which is below the human eye's flicker fusion threshold, making flickering clearly perceptible to users. Prolonged use can easily lead to severe visual fatigue, headaches, nausea, and other discomfort. For patients with photosensitive epilepsy, migraines, autism spectrum disorders, this flickering may even trigger seizures, posing significant safety risks and greatly limiting its application in core medical rehabilitation populations.

[0004] Second, there are issues with encoding capacity and signal-to-noise ratio (SNR). To increase the number of instructions, traditional methods require increasing the number of stimulus frequencies; however, the available high SNR frequencies that can induce effective SSVEP responses are limited. Insufficient frequency intervals can lead to response confusion and decreased recognition accuracy. Although some studies have attempted to use high-frequency stimuli (>60Hz) to avoid flickering, the SSVEP response amplitude decreases sharply with increasing frequency, resulting in an excessively low SNR, making reliable detection difficult.

[0005] Third, there are issues with system integration and application expansion. Existing solutions mostly focus on single-point improvements to stimulus paradigms or recognition algorithms, lacking end-to-end systematic integration from coding paradigms, stimulus presentation, signal processing to multi-scenario application control. This makes it difficult to meet the high requirements for system robustness, reliability, and practicality in medical rehabilitation scenarios.

[0006] Therefore, there is a need for a visual evoked brain-computer interface system and method based on high-frequency time-division multiplexing coding that is safe, seamless, has a large command throughput, and high recognition accuracy. Summary of the Invention

[0007] The main objective of this invention is to provide a visually evoked brain-computer interface system and method based on high-frequency time-division multiplexing coding, in order to solve the problems of visual fatigue, excessive coding capacity, and low recognition accuracy caused by low-frequency coding in existing brain-computer interfaces.

[0008] To achieve the above objectives, the present invention provides a visual evoked brain-computer interface system based on high-frequency time-division multiplexing coding, comprising: The visual stimulation module is located on the system display interface and includes multiple visual stimulation units.

[0009] The high-frequency time-division multiplexing coding module has multiple base frequency symbols and coding rules pre-stored; the frequency value range of the base frequency symbols is 60Hz to 90Hz; the coding rule is a time-division multiplexing (TDM) rule, which is used to combine at least two of the base frequency symbols in a specific time order to assign a unique coding sequence to each visual stimulus unit.

[0010] The signal acquisition and processing module is located in the user's brain. After being stimulated by the visual stimulation module, the signal acquisition and processing module acquires the user's EEG signals and preprocesses the acquired EEG signals.

[0011] The classification module, connected to the signal acquisition and processing module, predicts the encoding pattern of the preprocessed EEG signal after passing through the classification model module.

[0012] The application control module has a pre-set instruction mapping table, which is used to convert the encoding mode output by the classification module into control instructions for external devices or software.

[0013] This invention also provides a visually evoked brain-computer interface method based on high-frequency time-division multiplexing coding, specifically including the following steps: S1, generate multiple visual stimulus units on the screen, and use a high-frequency time-division multiplexing coding module to assign a unique coding sequence to each visual stimulus unit.

[0014] S2, the signal acquisition and processing module acquires the user's EEG signals and preprocesses the acquired EEG signals.

[0015] S3 uses a classification module to classify the preprocessed EEG signals, thereby predicting the encoding pattern of the EEG signals.

[0016] S4 converts the encoding mode output by the classification module into control instructions for external devices or software.

[0017] Furthermore, step S3 specifically includes the following steps: S3.1, Let the EEG signal be: ; in, This represents the number of electrode channels. This refers to a point in time.

[0018] S3.2 Extract the time-domain, frequency-domain, and spatial-domain features of the EEG signal.

[0019] S3.3 integrates time-domain features, frequency-domain features, and spatial-domain features.

[0020] S3.4, the fused features are input into a convolutional neural network to predict the encoding pattern of the EEG signal.

[0021] Furthermore, step S3.2 specifically includes the following steps: S3.2.1, Extracting temporal features of EEG signals using a one-dimensional convolutional neural network (1D-CNN): ; in, For the first Layer in time step The time-domain characteristics, For activation function, For the first Layer in time step The weight matrix, For the first Layer in time step The output characteristics, For the first Layer in time step The bias.

[0022] S3.2.2, Extracting the frequency domain features of the EEG signal: First, wavelet transform is used to convert the signal from the time domain to the frequency domain. ; in, The original time-domain signal, For the mother wavelet function, For the signal at scale ,Location The wavelet coefficients at that point.

[0023] Subsequently, the discretized time-frequency graph is input into a two-dimensional convolutional neural network (2D-CNN). ; in, For the first Frequency domain output characteristics of the layer For the first The frequency domain weights of the layer This is a two-dimensional convolution operation. For the first Frequency domain bias of the layer.

[0024] S3.2.3, Extracting spatial features using a graph convolutional network (GCN): ; in, No. Layered spatial characteristics, It is an adjacency matrix with self-connections. For degree matrix, For the first Layer weights.

[0025] Furthermore, step S3.3 specifically includes the following steps: S3.3.1, Calculate the temporal attention weights : ; in, This is the weight matrix. The time-domain feature matrix, This is the transpose of the attention weight vector. The hyperbolic tangent activation function is used. This is the temporal attention baseline bias.

[0026] S3.3.2, Calculate the frequency domain attention weights : ; in, This is the transpose of the frequency domain attention weight vector. The frequency domain weight matrix, For frequency domain characteristic matrix, This is the frequency domain attention baseline bias.

[0027] S3.3.3, Calculate the attention weights for spatial domain features. : ; in, This is the transpose of the spatial domain attention weight vector. The spatial domain weight matrix, For spatial domain characteristic matrix, This is the spatial domain attention baseline bias.

[0028] S3.3.4, Calculate the normalized weights: ; ; ; in, For normalized temporal attention weights, For normalized frequency domain attention weights, For the normalized spatial domain attention weights, It is an exponential function.

[0029] S3.3.5, fuses time-domain features, frequency-domain features, and spatial-domain features: ; in, The characteristics after fusion This is a feature fusion operator.

[0030] Furthermore, step S3.4 specifically includes the following steps: S3.4.1, input the fused features into the fully connected classification layer: ; in, To output the score vector, This is the weight matrix of the fully connected layer. This is the bias vector.

[0031] S3.4.2, the prediction results output during the training phase are: ; in, The first output of the fully connected layer The original score of the class, The total number of categories, For input, The output category labels; This represents the conditional probability.

[0032] S3.4.3, using cross-entropy loss The gap between the quantitative prediction results and the true labels: ; in, The total number of samples, The total number of categories, For the output of the first The category label of each sample, To take the logarithmic function, For the input of the first One sample, This is an indicator function.

[0033] S3.4.4, Backpropagation updates parameters, including: weights, biases, and convolutional layers.

[0034] S3.4.5, In the inference phase, the predicted results are output using the convolutional neural network model trained during the training phase: ,in, The index function for the maximum value.

[0035] The present invention has the following beneficial effects: This invention innovatively achieves "medical-grade safety": by setting the stimulation fundamental frequency to the 60-90Hz imperceptible band and combining it with TDM encoding, visual flicker perception is completely eliminated, fundamentally avoiding visual fatigue and the risk of photosensitive disease onset. This allows SSVEP-BCI technology to be safely applied to photosensitive patients such as those with epilepsy for the first time, which has significant clinical implications.

[0036] A leap in instruction throughput was achieved within a limited bandwidth: By adopting the TDM encoding mechanism, a large-scale unique encoding sequence of the order of N! can be constructed using N base frequencies, which perfectly solves the contradiction between the limited bandwidth of the high-frequency band and the large number of instructions required, and provides technical support for complex applications (such as virtual keyboards).

[0037] This invention ensures a high recognition rate under high-frequency stimulation: by employing advanced signal processing algorithms (such as individual template optimization) and making full use of the overall features of the TDM sequence for recognition, it effectively overcomes the problems of weak response and low signal-to-noise ratio of high-frequency SSVEP, and achieves stable recognition with high accuracy.

[0038] This invention provides a highly integrated solution: it is a complete closed-loop system from encoding, stimulation, recognition to control, which has been deeply optimized for medical rehabilitation scenarios, is highly practical, and is easy to promote and implement. Attached Figure Description

[0039] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart of a visual evoked brain-computer interface method based on high-frequency time-division multiplexing coding according to the present invention is shown. Detailed Implementation

[0040] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example 1 A visually evoked brain-computer interface system based on high-frequency time-division multiplexing coding, comprising: The visual stimulation module is located on the system display interface and includes multiple visual stimulation units. The high-frequency time-division multiplexing coding module has multiple pre-stored baseband symbols and coding rules; the frequency range of the baseband symbols is the high-frequency non-sensitive band from 60Hz to 90Hz; the coding rule is the time-division multiplexing (TDM) rule, which is used to combine at least two baseband symbols in a specific time order to assign a unique coding sequence to each visual stimulus unit. The signal acquisition and processing module is located in the user's brain. After being stimulated by the visual stimulation module, the signal acquisition and processing module acquires the user's EEG signals and preprocesses the acquired EEG signals. The classification module is connected to the signal acquisition and processing module. After the preprocessed EEG signal is passed through the classification model module, the coding pattern of the EEG signal is predicted. Through correlation analysis, pattern matching, or machine learning algorithms, the classification module can accurately determine which encoding pattern a user is looking at, thereby determining their intent. It does not rely on manually defined rules or simple similarity calculations, but rather allows the computer to learn a classification model from the data itself. Before the system is put into use, a model needs to be "trained" with a large amount of labeled data; machine learning algorithms (such as Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), and Convolutional Neural Networks (CNN)) automatically analyze this training data to find complex boundaries or feature combinations that can most effectively distinguish different encoding patterns. This process is adaptive and data-driven; when a new, unlabeled segment of a user's EEG signal is input, the trained model will directly predict which encoding pattern it is most likely to belong to.

[0042] The application control module has a pre-set instruction mapping table, which is used to convert the encoding mode output by the classification module into control instructions for external devices or software.

[0043] Example 2 like Figure 1 The visual evoked brain-computer interface method shown includes the following steps: S1 generates multiple visual stimulus units on the screen and assigns a unique coding sequence to each visual stimulus unit using a high-frequency time-division multiplexing coding module.

[0044] S2, the signal acquisition and processing module acquires the user's EEG signals and preprocesses the acquired EEG signals.

[0045] S3 uses a classification module to classify the preprocessed EEG signals, thereby predicting the encoding pattern of the EEG signals.

[0046] S4 converts the encoding mode output by the classification module into control instructions for external devices or software.

[0047] Specifically, step S3 includes the following steps: S3.1, Let the EEG signal be: ; in, This represents the number of electrode channels. For time points; S3.2 Extract the time-domain, frequency-domain, and spatial-domain features of the EEG signal.

[0048] S3.3 integrates time-domain features, frequency-domain features, and spatial-domain features.

[0049] S3.4, the fused features are input into a convolutional neural network to predict the encoding pattern of the EEG signal.

[0050] Specifically, step S3.2 includes the following steps: S3.2.1, Extracting temporal features of EEG signals using a one-dimensional convolutional neural network (1D-CNN): ; in, For the first Layer in time step The time-domain characteristics, For activation function, For the first Layer in time step The weight matrix, For the first Layer in time step The output characteristics, For the first Layer in time step The bias.

[0051] The above process extracts the waveform of the signal as it changes over time, reflecting the amplitude, fluctuation, and morphological characteristics of the signal.

[0052] S3.2.2, Extracting the frequency domain features of the EEG signal: First, wavelet transform is used to convert the signal from the time domain to the frequency domain. ; in, The original time-domain signal, For the mother wavelet function, For the signal at scale ,Location The wavelet coefficients at that point.

[0053] Transforming a signal from the time domain to the frequency domain and analyzing its energy distribution across different frequency components is the core of EEG analysis, because EEG activity is strongly correlated with specific frequency bands.

[0054] Subsequently, the discretized time-frequency graph is input into a two-dimensional convolutional neural network (2D-CNN). ; in, For the first Frequency domain output characteristics of the layer For the first The frequency domain weights of the layer This is a two-dimensional convolution operation. For the first Frequency domain bias of the layer.

[0055] The above process describes the forward propagation process of a two-dimensional convolutional neural network (2D-CNN) processing a time-frequency graph. This combination of CWT and CNN fully utilizes the time-frequency localization advantage of wavelet transform and the hierarchical feature learning capability of CNN, making it a powerful tool for processing non-stationary signals.

[0056] S3.2.3, Extracting spatial features using a graph convolutional network (GCN): ; in, No. Layered spatial characteristics, It is an adjacency matrix with self-connections. For degree matrix, For the first Layer weights.

[0057] Specifically, step S3.3 includes the following steps: S3.3.1, Calculate the temporal attention weights : ; in, This is the weight matrix. The time-domain feature matrix, This is the transpose of the attention weight vector. The hyperbolic tangent activation function is used. This is the temporal attention baseline bias.

[0058] Single time-domain, frequency-domain, or spatial-domain features often only characterize one aspect of an EEG signal. Fusing them can construct a more comprehensive and complementary signal representation, thereby improving the robustness and accuracy of the classification model.

[0059] S3.3.2, Calculate the frequency domain attention weights : ; in, This is the transpose of the frequency domain attention weight vector. The frequency domain weight matrix, For frequency domain characteristic matrix, This is the frequency domain attention baseline bias.

[0060] S3.3.3, Calculate the attention weights for spatial domain features. : ; in, This is the transpose of the spatial domain attention weight vector. The spatial domain weight matrix, For spatial domain characteristic matrix, This is the spatial domain attention baseline bias.

[0061] S3.3.4, Calculate the normalized weights: ; ; ; in, For normalized temporal attention weights, For normalized frequency domain attention weights, For the normalized spatial domain attention weights, It is an exponential function.

[0062] S3.3.5, fuses time-domain features, frequency-domain features, and spatial-domain features: ; in, The characteristics after fusion For feature fusion operators, , , Value range: [0,1].

[0063] Specifically, step S3.4 includes the following steps: S3.4.1, input the fused features into the fully connected classification layer: ; in, To output the score vector, This is the weight matrix of the fully connected layer. This is the bias vector.

[0064] S3.4.2, the prediction results output during the training phase are: ; in, The first output of the fully connected layer The original score of the class, The total number of categories, For input, The output category labels; This represents the conditional probability.

[0065] S3.4.3, using cross-entropy loss The gap between the quantitative prediction results and the true labels: ; in, The total number of samples, The total number of categories, For the output of the first The category label of each sample, To take the logarithmic function, For the input of the first One sample, This is an indicator function.

[0066] .

[0067] S3.4.4, Backpropagation updates parameters, including: weights, biases, and convolutional layers.

[0068] S3.4.5, In the inference phase, the predicted results are output using the convolutional neural network model trained during the training phase: ,in, The index function for the maximum value.

[0069] This invention provides a high-frequency SSVEP-BCI system and method that is safe, seamless, has a high command throughput, and high recognition accuracy. Through innovative coding design and system-level optimization, this invention fundamentally eliminates visual flicker, ensuring user safety and comfort, and significantly improves system performance, expanding its applications in fields such as intelligent rehabilitation and barrier-free communication.

[0070] The beneficial effects of the present invention will be explained below in conjunction with practical applications: The experimental conditions involved using a frequency range of 60Hz to 90Hz, with 2 rows x 3 columns of scintillation cells arranged via software. Participants were controlled to focus on different cells to receive stimulation within a specified time. A summary table of all participants' experimental results is provided, listing their performance, including stimulation time and accuracy.

[0071] As can be seen from the table, the accuracy of this invention is 0.426% higher than that of the existing method "Joint Frequency-Phase-Linear Frequency Modulation High-Frequency SSVEP Encoding and Decoding System and Method". This invention uses a convolutional neural network to predict whether the EEG signal belongs to a certain encoding mode, overcoming the problem of low recognition accuracy in the prior art.

[0072] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A visual evoked brain-computer interface system based on high frequency time-division multiplexing coding, characterized in that, include: The visual stimulation module is located on the system display interface and includes multiple visual stimulation units. The high-frequency time-division multiplexing coding module has multiple baseband symbols and coding rules pre-stored; the frequency value range of the baseband symbols is 60Hz to 90Hz; the coding rule is a time-division multiplexing (TDM) rule, which is used to combine at least two of the baseband symbols in a specific time order to assign a unique coding sequence to each visual stimulus unit. The signal acquisition and processing module is located in the user's brain. After being stimulated by the visual stimulation module, the signal acquisition and processing module acquires the user's EEG signals and preprocesses the acquired EEG signals. The classification module is connected to the signal acquisition and processing module. After the preprocessed EEG signal is passed through the classification model module, the coding pattern of the EEG signal is predicted. The application control module has a pre-set instruction mapping table, which is used to convert the encoding mode output by the classification module into control instructions for external devices or software.

2. A visual evoked brain-computer interface method based on high-frequency time-division multiplexing coding, applied to the visual evoked brain-computer interface system based on high-frequency time-division multiplexing coding as described in claim 1, characterized in that, Specifically, the steps include the following: S1, generate multiple visual stimulus units on the screen, and use a high-frequency time-division multiplexing coding module to assign a unique coding sequence to each visual stimulus unit; S2, the signal acquisition and processing module acquires the user's EEG signals and preprocesses the acquired EEG signals; S3 uses a classification module to classify the preprocessed EEG signals, thereby predicting the encoding pattern of the EEG signals; S4 converts the encoding mode output by the classification module into control instructions for external devices or software.

3. The visual evoked brain-computer interface method based on high-frequency time-division multiplexing coding according to claim 2, characterized in that, Step S3 specifically includes the following steps: S3.1, Let the EEG signal be: ; in, This represents the number of electrode channels. For time points; S3.2 Extract the time-domain, frequency-domain, and spatial-domain features of the EEG signal; S3.3, which fuses time-domain features, frequency-domain features, and spatial-domain features; S3.4, the fused features are input into a convolutional neural network to predict the encoding pattern of the EEG signal.

4. The visual evoked brain-computer interface method based on high-frequency time-division multiplexing coding according to claim 3, characterized in that, Step S3.2 specifically includes the following steps: S3.2.1, Extracting temporal features of EEG signals using a one-dimensional convolutional neural network (1D-CNN): ； in, For the first Layer in time step The time-domain characteristics, For activation function, For the first Layer in time step The weight matrix, For the first Layer in time step The output characteristics, For the first Layer in time step The bias; S3.2.2, Extracting the frequency domain features of the EEG signal: First, wavelet transform is used to convert the signal from the time domain to the frequency domain. ； in, The original time-domain signal, For the mother wavelet function, For the signal at scale ,Location Wavelet coefficients at; Subsequently, the discretized time-frequency graph is input into a two-dimensional convolutional neural network (2D-CNN). ； in, For the first Frequency domain output characteristics of the layer For the first The frequency domain weights of the layer This is a two-dimensional convolution operation. For the first Frequency domain bias of the layer; S3.2.3, Extracting spatial features using a graph convolutional network (GCN): ； in, No. Layered spatial characteristics, It is an adjacency matrix with self-connections. For degree matrix, For the first Layer weights.

5. The visual evoked brain-computer interface method based on high-frequency time-division multiplexing coding according to claim 3, characterized in that, Step S3.3 specifically includes the following steps: S3.3.1, Calculate the temporal attention weights : ； in, This is the weight matrix. The time-domain feature matrix, This is the transpose of the attention weight vector. The hyperbolic tangent activation function is used. For temporal attention baseline bias; S3.3.2, Calculate the frequency domain attention weights : ； in, This is the transpose of the frequency domain attention weight vector. The frequency domain weight matrix, For frequency domain characteristic matrix, For frequency domain attention baseline bias; S3.3.3, Calculate the attention weights for spatial domain features. : ； in, This is the transpose of the spatial domain attention weight vector. The spatial domain weight matrix, For the spatial domain characteristic matrix, Spatial domain attention baseline bias; S3.3.4, Calculate the normalized weights: ； ； ； in, For normalized temporal attention weights, For normalized frequency domain attention weights, For the normalized spatial domain attention weights, It is an exponential function; S3.3.5, fuses time-domain features, frequency-domain features, and spatial-domain features: ； in, The characteristics after fusion This is a feature fusion operator.

6. The visual evoked brain-computer interface method based on high-frequency time-division multiplexing coding according to claim 3, characterized in that, Step S3.4 specifically includes the following steps: S3.4.1, input the fused features into the fully connected classification layer: ； in, To output the score vector, This is the weight matrix of the fully connected layer. It is the bias vector; S3.4.2, the prediction results output during the training phase are: ； in, The first output of the fully connected layer The original score of the class, The total number of categories, For input, The output category labels; For conditional probability; S3.4.3, using cross-entropy loss The gap between the quantitative prediction results and the true labels: ； in, The total number of samples, The total number of categories, For the output of the first The category label of each sample, To take the logarithmic function, For the input of the first One sample, For indicator functions; S3.4.4, Backpropagation updates parameters, including: weights, biases, and convolutional layers; S3.4.5, In the inference phase, the predicted results are output using the convolutional neural network model trained during the training phase: ,in, The index function for the maximum value.

Images