Non-invasive brain-computer interface system based on deep learning and signal processing method thereof

Abstract

The invention relates to the technical field of brain-computer interfaces, and discloses a non-invasive brain-computer interface system based on deep learning and a signal processing method thereof, and the system comprises an EEG signal collection module, a signal preprocessing module, a deep learning model training module, and a decoding and control module. The EEG signal acquisition module is used for acquiring an EEG signal of a user; the signal preprocessing module is used for preprocessing the collected electroencephalogram signals, including noise suppression, filtering and feature extraction; the deep learning model training module is used for training a deep learning model and decoding the preprocessed electroencephalogram signal into a control instruction; the decoding module is used for decoding the electroencephalogram signals collected in real time into control instructions according to the trained model; and the control module is used for controlling external equipment according to the decoded control instruction. The method has the advantage of improving the performance and applicability of the non-invasive brain-computer interface.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface technology, and specifically to a non-invasive brain-computer interface system based on deep learning and a signal processing method thereof. Background Art

[0002] Brain-computer interface (BCI) technology is a technology that directly connects the human brain with external devices. It can convert brain signals into control commands to achieve human-computer interaction, allowing humans to directly control external devices through brain activities. At present, BCI technology has broad application prospects in medical rehabilitation, intelligent control, entertainment and other fields. Traditional BCI technology is mainly divided into two categories: invasive and non-invasive. Invasive BCI requires surgery to implant electrodes into the brain. Although the signal quality is high, there are greater surgical risks and ethical issues. Non-invasive BCI collects electroencephalogram (EEG) signals through scalp electrodes. Although the signal quality is low, it is safer and more acceptable.

[0003] In recent years, deep learning has achieved remarkable results in image recognition, speech recognition and other fields. Its powerful feature extraction and pattern recognition capabilities have brought new opportunities for BCI signal processing. However, the nonlinearity, low signal-to-noise ratio and time-varying nature of EEG signals make deep learning-based signal processing methods face many challenges in BCI applications. Therefore, it is necessary to design a non-invasive brain-computer interface system and its signal processing method based on deep learning to improve the performance and applicability of non-invasive brain-computer interfaces. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a non-invasive brain-computer interface system based on deep learning and a signal processing method thereof, which has the advantages of improving the performance and applicability of the non-invasive brain-computer interface and solves the problems in the above-mentioned background technology.

[0005] In order to achieve the above-mentioned purpose of improving the performance and applicability of non-invasive brain-computer interface, the present invention provides the following technical solutions: a non-invasive brain-computer interface system based on deep learning and a signal processing method thereof, comprising an EEG signal acquisition module, a signal preprocessing module, a deep learning model training module, a decoding module and a control module;

[0006] The EEG signal acquisition module collects the user's EEG signal;

[0007] The signal preprocessing module is used to preprocess the collected EEG signals, including noise suppression, filtering and feature extraction;

[0008] The deep learning model training module is used to train the deep learning model and decode the preprocessed EEG signals into control instructions;

[0009] The decoding module is used to decode the real-time collected EEG signals into control instructions according to the trained model;

[0010] The control module is used to control the external device according to the decoded control instruction.

[0011] Preferably, the EEG signal acquisition module further comprises:

[0012] Select EEG sensor: Select EEG sensor according to application scenario and user needs;

[0013] Determine the electrode placement location: Select the electrode placement location based on EEG signal characteristics and application scenarios;

[0014] Set the signal acquisition frequency: capture EEG signal characteristics;

[0015] Design for noise immunity: use shielded cables, isolation amplifiers.

[0016] Preferably, the signal preprocessing module further comprises:

[0017] Noise removal: Use noise suppression algorithm to remove electrooculography, electromyography and power frequency interference in EEG signals;

[0018] Filtering: Use filters to remove unwanted frequency components from the EEG signal;

[0019] Feature extraction: Extract features from EEG signals, including time domain features, frequency domain features, and time-frequency domain features.

[0020] Preferably, the deep learning model training module further includes:

[0021] Model selection: Select a deep learning model based on the application scenario and data characteristics;

[0022] Prepare training data: collect a large amount of EEG signal data and annotate it;

[0023] Training model: Use training data to train the deep learning model, learn the characteristics of EEG signals, and establish a mapping relationship from characteristics to control instructions;

[0024] Optimize model: Use model optimization technology to improve the generalization and robustness of the model.

[0025] Preferably, the decoding and control module further comprises:

[0026] Real-time decoding: Use the trained deep learning model to decode the real-time collected EEG signals and generate control instructions;

[0027] Control interface: Design the control interface to pass the decoded control instructions to external devices;

[0028] Feedback mechanism: Establish a feedback mechanism to transmit feedback information from external devices to the system for updating the model and optimizing the control strategy.

[0029] A non-invasive brain-computer interface signal processing method based on deep learning, characterized in that it comprises the following steps:

[0030] S1: Select the EEG sensor, determine the electrode placement, and set the signal acquisition frequency;

[0031] Preferably, S1 further includes considering signal quality, wearing comfort, cost and applicable scenarios when selecting an EEG sensor, placing electrodes according to the international 10-20 system, and the acquisition frequency must match the target signal frequency and data processing capability.

[0032] S2: Perform anti-noise design to determine whether the data quality is qualified;

[0033] Preferably, the S2 further includes if the data quality is qualified, performing noise filtering on the collected data, removing unnecessary frequency components through the filter, and extracting characteristic information from the filtered signal;

[0034] Noise removal: ICA, empirical mode decomposition and wavelet transform methods are used to remove different types of noise, which is implemented using the MNE-Python library;

[0035] Filtering: Apply bandpass and bandstop filters to remove interference at specific frequencies. The SciPy library can be used to design filters.

[0036] Feature extraction: Select the feature extraction method according to the research objectives and implement it using Python library;

[0037] The collected data is input into the deep learning model after denoising, filtering and feature extraction;

[0038] If the data is not qualified, the process ends.

[0039] S3: Select a deep learning model based on data characteristics, divide the data into training, validation, and test sets, and judge whether the evaluation model is qualified;

[0040] Preferably, S3 further comprises:

[0041] Based on the extracted features, select CNN, RNN or LSTM model, divide the data into training set, validation set and test set, train and evaluate the model, use the training data to train the selected model, evaluate the eligibility of the model, evaluate the performance of the model based on the validation data, and decide whether the model is qualified. If it is not qualified, return to the model selection step to reselect and train the model. If the model is qualified, decode the real-time collected EEG signals, design a control interface for interacting with the system, establish a feedback mechanism for adjusting the system and model, and finally, check whether the model performance meets the requirements: if it does, the process is completed; if it does not, the process ends.

[0042] S4: Use the trained model to decode EEG signals in real time, select the control interface according to the application scenario, and provide visual, auditory or tactile feedback.

[0043] Preferably, the S4 further comprises:

[0044] Use the set EEG sensors to collect the user's EEG signals in real time, decode the collected EEG signals in real time through the trained model, extract the user's intention, brain wave pattern or specific cognitive state, identify the need for decoding signals according to the actual application scenario, select the control interface, and provide visual, auditory or tactile feedback based on the decoding results and operation requirements. Based on the real-time feedback of user operations, the system can adjust itself according to the current state.

[0045] Compared with the prior art, the present invention provides a non-invasive brain-computer interface system based on deep learning and a signal processing method thereof, which has the following beneficial effects:

[0046] 1. By adopting a deep learning model, it is possible to learn the complex features in EEG signals, improve the decoding accuracy of signals, and achieve more accurate control instruction generation.

[0047] 2. Choose lightweight deep learning models, such as convolutional neural networks (CNN) or recurrent neural networks (RNN), to reduce model training time and hardware resource requirements, and facilitate practical applications.

[0048] 3. By using adaptive learning mechanisms, the generalization ability of the model is improved so that it can adapt to different users and different tasks, reducing the demand for training data.

[0049] 4. It can realize a more natural and convenient human-computer interaction method and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The method system architecture of the present invention;

[0051] Figure 2 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The present invention provides a technical solution: a non-invasive brain-computer interface signal processing method based on deep learning, comprising the following steps:

[0054] S1: Select the EEG sensor, determine the electrode placement, and set the signal acquisition frequency;

[0055] S2: Perform anti-noise design to determine whether the data quality is qualified;

[0056] S3: Select a deep learning model based on data characteristics, divide the data into training, validation, and test sets, and judge whether the evaluation model is qualified;

[0057] S4: Use the trained model to decode EEG signals in real time, select the control interface according to the application scenario, and provide visual, auditory or tactile feedback.

[0058] During the data acquisition process, real-time signal quality monitoring and adaptive filtering are performed, and the data is processed by improved noise suppression and artifact removal technology, and then multi-channel synchronous acquisition and data fusion are performed, and automatic artifact identification and marking are performed.

[0059] Intelligent electrode selection and dynamic configuration: Automatically adjust the position of the electrode according to the user's head shape and the location of the target brain area to ensure the best signal coverage and acquisition effect. This design can reduce manual intervention and improve acquisition accuracy. Combining the advantages of dry electrodes and wet electrodes, hybrid electrodes are used in the same device to improve signal quality and wearing comfort. Wet electrodes are used for high-precision signal acquisition, and dry electrodes are used to reduce device complexity and simplify operation procedures.

[0060] Real-time signal quality monitoring and adaptive filtering: A signal quality detection module is added during the data acquisition process to monitor the stability and signal-to-noise ratio of the signal in real time. Once excessive noise or signal quality degradation is detected, the acquisition parameters can be automatically adjusted or an alarm can be issued to alert the operator. An adaptive bandpass filtering algorithm is introduced to dynamically adjust the filter parameters according to the characteristics of the real-time collected EEG signal to optimize the clarity and reliability of the signal, thereby improving the anti-interference ability.

[0061] Improved noise suppression and artifact removal technology: The improved independent component analysis algorithm can more effectively separate artifacts in EEG signals, such as electrooculography, electrocardiography, and electromyography interference, further improving the purity of the signal. Develop a multi-source interference detection system that combines multi-sensor data, such as environmental noise sensors and accelerometers, to identify and remove non-brain-derived signals such as electromagnetic interference and motion artifacts.

[0062] Multi-channel synchronous acquisition and data fusion: Use multiple acquisition channels to simultaneously record the activities of different brain regions, and use high-precision time synchronization technology to ensure that the data of all channels are completely synchronized during the acquisition process, so that the spatiotemporal features can be accurately extracted in subsequent analysis. Multimodal data fusion analysis of EEG signals and other physiological signals, such as electrocardiogram, electrodermal, and respiratory signals, can improve the depth of understanding of EEG signal changes and the accuracy of analysis.

[0063] Automatic artifact recognition and marking system: Develop an artifact recognition system based on machine learning, which automatically detects and marks interference artifacts in EEG signals, such as eye movements, muscle activity, and environmental noise, through training models, thereby reducing the workload of manual inspection and improving the accuracy of signal analysis. During the acquisition process, the system can automatically adjust the acquisition settings or prompt the user to adjust the wearing method according to the artifact recognition results to minimize the generation of artifacts.

[0064] After data collection, data preprocessing is performed. During data preprocessing, adaptive denoising capability is improved, precise feature extraction technology is used, and then real-time filtering optimization is performed, and the anti-noise signal is enhanced.

[0065] Improved adaptive denoising capabilities: A combination of multiple denoising algorithms, such as ICA, EMD, and wavelet transform, are used to adaptively select the best method to remove different types of noise based on the characteristics of the signal. This adaptive denoising method works better when dealing with complex, multi-source interference signals, significantly reducing the need for manual parameter adjustment. EMD has unique advantages in dealing with nonlinear and non-stationary signals. It can decompose complex signals into different internal model functions, thereby effectively extracting and removing noise, which is difficult to achieve with traditional linear filtering methods.

[0066] Precise feature extraction technology: By combining feature extraction techniques in the time domain, frequency domain, and space domain, the multi-dimensional description capability of EEG signal features has been significantly improved. For example, the power spectral density combined with spatial filtering, such as the use of common spatial patterns (CSP), enables the model to accurately extract features related to specific brain area activities in multi-channel EEG data. This multi-dimensional feature extraction method provides richer information for deep learning models and greatly improves classification accuracy. Feature selection mechanisms commonly used in deep neural networks are added to the feature extraction process, such as feature screening based on importance weights, so that the model only focuses on the most discriminative features, avoiding information redundancy and overfitting.

[0067] Real-time filter optimization: The filter design not only uses traditional bandpass and bandstop filters, but also combines adaptive filtering algorithms to dynamically adjust filter parameters according to the characteristics of the input signal. This dynamic filter design greatly improves signal clarity, especially when dealing with sudden noise or environmental interference. The signal processing module in the SciPy library is used to optimize the filter implementation, so that it can minimize delays while ensuring signal quality, and meet the needs of real-time signal processing.

[0068] Noise-resistant signal enhancement: During data preprocessing, nonlinear signal processing technology is used to improve the processing capability of low signal-to-noise ratio signals, which enables EEG signals collected in complex environments to maintain high-quality feature extraction. This method particularly solves the problem that traditional linear filtering is not effective in processing low signal-to-noise ratio signals. Independent component analysis, when combined with wavelet transform, can effectively separate signal components from different sources and perform refined denoising to ensure the purity of input data for deep learning models.

[0069] The preprocessed data is input into the deep learning model. Depending on the data type, deep learning model training can select any of CNN, RNN and LSTM.

[0070] CNN is suitable for signal data with significant spatiotemporal characteristics, such as characteristic patterns and spectrograms of local brain regions. When EEG signals are converted into two-dimensional or three-dimensional images, such as time-frequency graphs, CNN can effectively extract spatial features and local patterns. For cases where data has strong local correlation, such as time-frequency analysis of EEG, CNN performs better.

[0071] RNN is suitable for time series data, especially for scenarios where long-term dependencies are not significant. When EEG signals have a certain sequential dependency in the time dimension, RNN can capture the characteristics of the signal in the time dimension through its cyclic structure, but its ability to capture long-term dependencies is limited.

[0072] LSTM is suitable for processing long-term dependent time series signal data. When EEG signals show strong time dependence or long-term trends, LSTM can better capture the long-term dependent features in the signal through its memory unit and gating mechanism. This is particularly effective when complex time series patterns need to be predicted.

[0073] The data is divided into training, validation, and test sets, appropriate loss functions and optimizers are used, and the training process is monitored to optimize the parameters.

[0074] Loss function: An important concept in machine learning and deep learning used to evaluate the difference between model predictions and true results. It quantifies the model's prediction error and guides the optimization of the model during training. The smaller the value of the loss function, the closer the model's prediction is to the true value.

[0075] Cross entropy loss: Applicable to classification problems, especially multi-category classification tasks. In the EEG signal classification scenario, if the task is to classify multiple categories of states, cross entropy loss is a common choice.

[0076] Mean square error: Mainly used for regression tasks. When the output of the model is a continuous value, such as EEG signal amplitude prediction, MSE is a suitable choice.

[0077] Binary cross entropy: used for binary classification problems, such as detecting the occurrence of a specific state or event in an EEG signal.

[0078] Optimizer: In machine learning and deep learning, it is responsible for adjusting the parameters of the model to minimize the value of the loss function, thereby improving the performance of the model. It iteratively updates the parameters to find the best parameter combination that minimizes the loss function. The selection and setting of the optimizer have an important impact on the training effect and convergence speed of the model. The optimizer type can be any of SGD, Adam, and RMSprop.

[0079] SGD: Suitable for simple model training on large datasets, but may converge slowly.

[0080] Adam: It is often used to deal with non-stationary targets. It has good convergence speed and stability and is suitable for most deep learning tasks.

[0081] RMSprop: It is suitable for processing non-stationary targets and can adaptively adjust gradient updates in different directions. It is often used to process sequence data, such as EEG signal time series.

[0082] The performance of the model on the test set is evaluated through indicators such as accuracy and F1 score.

[0083] Eligibility criteria: If the accuracy rate exceeds 80% or 90%, it can generally be considered that the model performs well in most scenarios.

[0084] Failure criteria: An accuracy rate below 70% usually indicates that the model has serious misclassification problems when identifying EEG signal features and needs further optimization.

[0085] Passing criteria: When the F1 score is 0.8 and above, it indicates that the model has achieved a good balance between precision and recall.

[0086] Failure criteria: When the F1 score is lower than 0.7, it usually indicates that the model has significant deviations when predicting positive and negative examples, and the model parameters or preprocessing steps need to be readjusted.

[0087] The above deep learning models also include adaptive model architecture selection, data-driven hyperparameter optimization, and multi-scale feature extraction.

[0088] Adaptive model architecture selection: Dynamically select CNN, RNN, LSTM and other models according to data characteristics, and combine model combination strategies to automatically configure the best model architecture for different EEG tasks, such as classification and regression. This approach significantly improves the generalization ability and feature learning effect of the model, especially in applications under multiple task scenarios. Integrating multiple models, such as the CNN-LSTM hybrid model for joint modeling of spatiotemporal features, solves the limitations of a single model in complex EEG signal processing.

[0089] Data-driven hyperparameter optimization: The hyperparameter tuning strategy based on Bayesian optimization or genetic algorithm is introduced to make the hyperparameter selection in the model training process, such as learning rate, batch size, network depth, etc., more efficient, reducing the time of manual trial and error. In view of the imbalance and noise problems of EEG signal data, the loss function is optimized so that the model can adapt to multiple types of unbalanced data sets during the learning process, improving the robustness of the model.

[0090] Multi-scale feature extraction: Multi-scale convolution kernels are added to the model structure design. By extracting features from signals at different time scales, the multi-scale features of the signal are fully utilized. This design greatly enhances the model's ability to perceive complex EEG signal patterns. The time window sliding technology is used to segment the signal, allowing the model to capture the dynamic changes of the EEG signal and improve the sensitivity of emergency detection.

[0091] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0092] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A non-invasive brain-computer interface system based on deep learning, characterized in that: It includes EEG signal acquisition module, signal preprocessing module, deep learning model training module, decoding and control module; The EEG signal acquisition module collects the user's EEG signal; The signal preprocessing module is used to preprocess the collected EEG signals, including noise suppression, filtering and feature extraction; The deep learning model training module is used to train the deep learning model and decode the preprocessed EEG signals into control instructions; The decoding module is used to decode the real-time collected EEG signals into control instructions according to the trained model; The control module is used to control the external device according to the decoded control instruction.

2. The non-invasive brain-computer interface system based on deep learning according to claim 1 is characterized in that: The EEG signal acquisition module further comprises: Select EEG sensor: Select EEG sensor according to application scenario and user needs; Determine the electrode placement location: Select the electrode placement location based on EEG signal characteristics and application scenarios; Set the signal acquisition frequency: capture EEG signal characteristics; Design for noise immunity: use shielded cables, isolation amplifiers.

3. The non-invasive brain-computer interface system based on deep learning according to claim 1 is characterized in that: The signal preprocessing module further comprises: Noise removal: Use noise suppression algorithm to remove electrooculography, electromyography and power frequency interference in EEG signals; Filtering: Use filters to remove unwanted frequency components from the EEG signal; Feature extraction: Extract features from EEG signals, including time domain features, frequency domain features, and time-frequency domain features.

4. The non-invasive brain-computer interface system based on deep learning according to claim 1 is characterized in that: The deep learning model training module further includes: Model selection: Select a deep learning model based on the application scenario and data characteristics; Prepare training data: collect a large amount of EEG signal data and annotate it; Training model: Use training data to train the deep learning model, learn the characteristics of EEG signals, and establish a mapping relationship from characteristics to control instructions; Optimize model: Use model optimization technology to improve the generalization and robustness of the model.

5. The non-invasive brain-computer interface system based on deep learning according to claim 1, characterized in that: The decoding and control module further comprises: Real-time decoding: Use the trained deep learning model to decode the real-time collected EEG signals and generate control instructions; Control interface: Design the control interface to pass the decoded control instructions to external devices; Feedback mechanism: Establish a feedback mechanism to transmit feedback information from external devices to the system for updating the model and optimizing the control strategy.

6. A non-invasive brain-computer interface signal processing method based on deep learning, characterized in that: The following steps are involved: S1: Select the EEG sensor, determine the electrode placement, and set the signal acquisition frequency; S2: Perform anti-noise design to determine whether the data quality is qualified; S3: Select a deep learning model based on data characteristics, divide the data into training, validation, and test sets, and judge whether the evaluation model is qualified; S4: Use the trained model to decode EEG signals in real time, select the control interface according to the application scenario, and provide visual, auditory or tactile feedback.

7. The non-invasive brain-computer interface signal processing method based on deep learning according to claim 6 is characterized in that: The S1 further includes considering signal quality, wearing comfort, cost and applicable scenarios when selecting an EEG sensor, placing electrodes according to the international 10-20 system, and the acquisition frequency must match the target signal frequency and data processing capability.

8. The non-invasive brain-computer interface system based on deep learning and the signal processing method thereof according to claim 6, characterized in that: The S2 further includes if the data quality is qualified, performing noise filtering on the collected data, removing unnecessary frequency components through the filter, and extracting characteristic information from the filtered signal; Noise removal: ICA, empirical mode decomposition and wavelet transform methods are used to remove different types of noise, which are implemented using the MNE-Python library; Filtering: Apply bandpass and bandstop filters to remove interference at specific frequencies. SciPy libraries can be used to design filters. Feature extraction: Select the feature extraction method according to the research objectives and implement it using Python library; The collected data is input into the deep learning model after denoising, filtering and feature extraction; If the data is not qualified, the process ends.

9. The non-invasive brain-computer interface system based on deep learning and the signal processing method thereof according to claim 6, characterized in that: The S3 further comprises: Based on the extracted features, select CNN, RNN or LSTM model, divide the data into training set, validation set and test set, train and evaluate the model, use the training data to train the selected model, evaluate the eligibility of the model, evaluate the performance of the model based on the validation data, and decide whether the model is qualified. If it is not qualified, return to the model selection step to reselect and train the model. If the model is qualified, decode the real-time collected EEG signals, design a control interface for interacting with the system, establish a feedback mechanism for adjusting the system and model, and finally, check whether the model performance meets the requirements: if it does, the process is completed; if it does not, the process ends.

10. The non-invasive brain-computer interface system based on deep learning and the signal processing method thereof according to claim 6, characterized in that: The S4 further comprises: Use the set EEG sensors to collect the user's EEG signals in real time, decode the collected EEG signals in real time through the trained model, extract the user's intention, brain wave pattern or specific cognitive state, identify the need for decoding signals according to the actual application scenario, select the control interface, and provide visual, auditory or tactile feedback based on the decoding results and operation requirements. Based on the real-time feedback of user operations, the system can adjust itself according to the current state.

Images