Brain-computer interface efficient decoding method and device based on signal quality pre-screening, medium and product

Abstract

The invention relates to the technical field of brain-computer interfaces, and discloses a brain-computer interface efficient decoding method and device based on signal quality pre-screening, a medium and a product. The method comprises the following steps: firstly, extracting time domain, frequency domain and time-frequency domain characteristics of electroencephalogram signals through a signal quality judgment module, evaluating by using models such as a support vector machine, and pre-screening the signals according to a quality threshold value; secondly, through a feature extraction and selection module, independent component analysis is carried out on qualified quality signals, and a key feature subset is selected by using a Relief-F algorithm; and then, through a classification decision module, a long-short term memory network is combined with an adaptive learning rate and a regularization method to carry out classification. And finally, dynamically updating parameters and models of each module according to user feedback and monitoring results through an adaptive adjustment module. The method can at least solve the technical problems of low decoding efficiency and low accuracy caused by unstable quality of electroencephalogram signals.

Description

Technical Field

[0001] This application relates to the field of brain-computer interface technology, and in particular to a novel EEG signal decoding algorithm framework for a brain-computer interface speller. Background Technology

[0002] In recent years, a series of groundbreaking advancements have been achieved in the field of brain-computer interfaces (BCIs). For example, Nature magazine reported on thought-based speech synthesis and thought-based handwriting in 2019 and 2021, respectively. Advances in paradigms and algorithms have increased the information transmission rate of BCIs to over 7 bits per second, gradually approaching that of traditional interactive interfaces such as keyboards and mice. However, it is important to note that the aforementioned studies all employed invasive acquisition methods, such as flexible electrodes. While this approach achieves higher signal-to-noise ratios and spatial resolution, the associated surgical risks and maintenance costs make it difficult to gain widespread acceptance and large-scale application in the near future. In contrast, non-invasive methods, such as electroencephalography (EEG), offer advantages such as low risk, low cost, and ease of maintenance, making them more practically significant at present.

[0003] Brain-computer spelling devices are one of the most typical applications of non-invasive brain-computer interfaces, helping users input text or spell words via EEG signals. Following predetermined rules (also known as brain-computer interface paradigms), users select different letters or words by performing different mental tasks to stimulate specific brain activity patterns. This technology is particularly helpful for people with limited muscle control due to stroke, brain injury, or motor neuron disease. Through brain-computer spelling devices, they can communicate with the outside world more directly, improving their quality of life and autonomy.

[0004] For ease of explanation, the steady-state visual-evoked potential (SSVEP) paradigm will be used as an example. Figure 1 The basic workflow of a brain-computer interface (BCI) spelling device is demonstrated. Time is strictly divided into alternating, fixed-length spelling slots and rest slots. Users are instructed to perform specific mental tasks to complete the spelling operation during the spelling slots, while remaining calm, receiving system feedback, and preparing for the next spelling slot during the rest slots. At the end of each spelling slot, the system analyzes the synchronously acquired EEG signals to make a decision and execute the corresponding operation, either adding a new character to the queue or deleting a character from the end of the queue. For BCI spelling devices based on different paradigms, the design of other components remains consistent, except for the differences in mental tasks and decoding algorithms.

[0005] However, the quality of EEG signals is unstable, and errors are inevitable when decoding them. Once an error occurs, the user needs to delete the erroneous character using "backspace," but during this process, the user does not output any valid characters, resulting in wasted time and reduced information transmission rate of the brain-computer interface. Summary of the Invention

[0006] One objective of this application is to provide a method, device, medium, and product for efficient decoding of brain-computer interfaces based on signal quality pre-screening, which at least addresses the problems existing in the aforementioned brain-computer interface spellers.

[0007] To achieve the above objectives, some embodiments of this application provide the following aspects:

[0008] Firstly, some embodiments of this application provide an efficient brain-computer interface decoding method based on signal quality pre-screening. This method involves: extracting time-domain, frequency-domain, and time-frequency-domain features from acquired EEG signals using a signal quality judgment module; constructing a signal quality assessment model using support vector machine (SVM) or other algorithms; judging the quality of the EEG signal based on a set quality threshold to determine whether to send the signal to subsequent modules; further separating independent components and extracting features from the EEG signals that have passed quality judgment using a feature extraction and selection module; selecting extracted features using algorithms such as Relief-F, calculating feature weights, and selecting a subset of features with higher weights to reduce feature dimensionality; classifying the EEG signals after feature extraction and selection using a classification decision module employing algorithms such as Long Short-Term Memory (LSTM) networks; during training, using an adaptive learning rate adjustment strategy combined with L1 and L2 regularization methods to prevent model overfitting and outputting accurate EEG signal classification results; and monitoring the accuracy and stability of the classification results in real time using an adaptive adjustment module, periodically collecting new EEG signal data based on user feedback to update and optimize the signal quality assessment model and classification model.

[0009] Secondly, some embodiments of this application also provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method described above.

[0010] Thirdly, some embodiments of this application also provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the method described above.

[0011] Fourthly, some embodiments of this application also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0012] Compared with related technologies, the solution provided in this application designs a novel EEG signal decoding algorithm. The decoding algorithm is a classification model, while the classification output of traditional algorithms is generally the instruction of the brain-computer interface (BCI), such as different characters or operations like "backspace" in a BCI speller. To further improve the information transmission rate of the BCI and increase the number of characters that the BCI speller can output per unit time, this invention proposes a novel EEG signal decoding algorithm framework for BCI spellers. This framework, in addition to outputting the classification result of the EEG signal like traditional algorithms, can also pre-judge the quality of the EEG signal. Therefore, if the EEG signal quality is judged to be poor, the classification result can be omitted, thus avoiding the necessary backspace and other extra operations after an error. Ultimately, this achieves the goal of improving the spelling speed and enhancing the brain-computer information transmission capability. Attached Figure Description

[0013] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise specified, the figures in the drawings are not to be limited by scale.

[0014] Figure 1 A basic workflow diagram of a brain-computer spelling device is provided for some embodiments;

[0015] Figure 2 A flowchart of the proposed novel brain-computer interface spelling system is provided for some embodiments;

[0016] Figure 3 An exemplary structural diagram of an electronic device is provided for some embodiments. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] This novel EEG signal decoding algorithm framework mainly consists of five core modules: data acquisition and preprocessing, signal quality assessment, feature extraction and selection, classification decision, and adaptive adjustment. The core of the framework lies in first assessing the quality of the EEG signal before classifying and outputting it, thus avoiding errors caused by low-quality signals and improving the information transmission rate of the brain-computer interface speller.

[0019] This embodiment provides a high-efficiency decoding method for brain-computer interfaces based on signal quality pre-screening, including:

[0020] The signal quality assessment module extracts time-domain, frequency-domain, and time-frequency-domain features from the acquired EEG signals, such as mean, variance, power spectral density, and wavelet coefficient energy. A signal quality evaluation model is constructed using a support vector machine algorithm or other algorithms. Based on a set quality threshold, the EEG signal quality is assessed to determine whether the signal should be sent to subsequent modules.

[0021] Through a feature extraction and selection module, for EEG signals that have passed quality assessment, independent component analysis and other methods are used to further separate independent components and extract features. Algorithms such as Relief-F are used to select extracted features, calculate feature weights, and select a subset of features with higher weights to reduce feature dimensionality.

[0022] The classification decision module employs algorithms such as Long Short-Term Memory (LSTM) networks to classify EEG signals after feature extraction and selection. During training, adaptive learning rate adjustment strategies, such as Adagrad and Adadelta, are used, combined with L1 and L2 regularization methods to prevent model overfitting, resulting in accurate EEG signal classification results.

[0023] The adaptive adjustment module monitors the accuracy and stability of classification results in real time, dynamically adjusting signal quality thresholds, feature selection parameters, and classification model parameters based on user feedback such as input error rate and operation time. New EEG signal data is collected periodically to update and optimize the signal quality assessment and classification models.

[0024] The following provides a detailed explanation of each step:

[0025] (I) Data Acquisition and Preprocessing

[0026] 1. Data Acquisition: High-precision, multi-channel EEG signal acquisition equipment is used to ensure comprehensive and accurate capture of the weak electrical signals generated by brain activity. The equipment must have a high sampling rate (e.g., not less than 1000Hz) and high resolution to obtain rich EEG information.

[0027] 2. Preprocessing procedure

[0028] 1. Filtering: Use a bandpass filter to filter out noise such as power frequency interference (e.g., 50Hz or 60Hz), electromyography (EMG) interference, and electrooculography (EOG) interference from the EEG signal. Set an appropriate passband range, such as 0.5-100Hz, to retain effective frequency components related to brain cognitive activity.

[0029] 2. Rereference: The acquired EEG signals are rereferenced using methods such as average reference or common reference to eliminate potential differences between different electrodes and improve the comparability and stability of the signals.

[0030] 3. Baseline correction: By calculating the average value of the signal over an initial period of time as the baseline, the subsequent signal is subtracted from this baseline value to eliminate DC drift in the signal and make the signal fluctuate around zero potential.

[0031] (ii) Signal quality assessment

[0032] 1. Feature Extraction

[0033] 1. Time-domain characteristics: Calculate the mean, variance, peak-to-peak value, and zero-crossing rate of the EEG signal. The mean reflects the average level of the signal, the variance reflects the degree of signal fluctuation, the peak-to-peak value represents the maximum amplitude range of the signal, and the zero-crossing rate reflects the rate of change of the signal.

[0034] 2. Frequency Domain Characteristics: The EEG signal is converted to the frequency domain using Fast Fourier Transform (FFT), and the power spectral density of different frequency bands (e.g., delta band 0-4Hz, theta band 4-8Hz, alpha band 8-13Hz, beta band 13-30Hz, and gamma band 30-100Hz) is extracted. The power spectral density of different frequency bands reflects different physiological states and cognitive activities of the brain.

[0035] 3. Time-Frequency Domain Features: Wavelet transform is used to perform time-frequency analysis on EEG signals, extracting features such as energy and entropy of the wavelet coefficients. Wavelet transform can simultaneously provide time and frequency information of the signal, providing a more comprehensive description of the time-varying characteristics of EEG signals.

[0036] 2. Quality Assessment Model

[0037] 1. A signal quality assessment model was constructed using the Support Vector Machine (SVM) algorithm. SVM is a supervised learning model that can find the optimal classification hyperplane in a high-dimensional space to effectively distinguish between different categories. In EEG signal quality assessment, it can classify EEG signals into high-quality and low-quality categories based on extracted multi-domain features.

[0038] 2. For the selection of kernel function, radial basis function (RBF) is adopted, which has good performance in dealing with nonlinear separable problems and can adapt to the complex feature distribution of EEG signals.

[0039] 3. The SVM model was trained using a large number of well-labeled high-quality and low-quality EEG signal samples. During training, a grid search method combined with cross-validation was used to optimize the model parameters, such as the penalty factor C and the kernel width parameter γ. The penalty factor C controls the degree of punishment for classification errors; the larger the C value, the heavier the punishment for misclassification. γ affects the shape of the radial basis function, thus affecting the model's complexity and generalization ability.

[0040] 3. Decision-making mechanism: A quality threshold is set. When the quality assessment score of the model output is higher than the threshold, the EEG signal is judged to be of good quality and the signal is sent to the subsequent feature extraction and selection module. When the quality assessment score is lower than the threshold, the EEG signal is judged to be of poor quality and no further classification processing is performed to avoid erroneous output.

[0041] (III) Feature Extraction and Selection

[0042] 1. Further Feature Extraction: For EEG signals judged by signal quality, Independent Component Analysis (ICA) is used to separate different independent components and extract features of each independent component, such as energy and frequency. ICA can effectively separate components from different sources in EEG signals, such as activity in different brain regions and electrooculography (EOG), thus improving the specificity of features.

[0043] 2. Feature Selection: The Relief-F algorithm is used to select extracted features. The Relief-F algorithm is an instance-based feature selection method that evaluates the importance of features to classification by calculating feature weights, and selects features with higher weights as the final feature subset, thereby reducing feature dimensionality and improving classification efficiency.

[0044] (iv) Categorized decision-making

[0045] 1. Classification Model: Long Short-Term Memory (LSTM) network is used for classifying EEG signals. LSTM is a special type of recurrent neural network that can effectively process sequential data and has the ability to remember long-term information, making it suitable for processing time-series data such as EEG signals.

[0046] 2. Training and Optimization: The LSTM model is trained using labeled, high-quality EEG signal samples. During training, an adaptive learning rate adjustment strategy (such as Adagrad, Adadelta, etc.) is employed to dynamically adjust the learning rate based on the model's training progress, accelerating convergence and improving the model's generalization ability. Simultaneously, regularization methods (such as L1 and L2 regularization) are used to prevent overfitting.

[0047] (v) Adaptive adjustment

[0048] 1. Parameter Adjustment: The system monitors the accuracy and stability of the classification results in real time, and dynamically adjusts the signal quality threshold, feature selection parameters, and classification model parameters based on user feedback (such as input error rate, operation time, etc.). For example, when the error rate is high, the signal quality threshold is appropriately increased to reduce the processing of low-quality signals; when the operation time is too long, the parameters of the classification model are adjusted to improve the classification speed.

[0049] 2. Model Updates: New EEG signal data is collected regularly to update and optimize the signal quality assessment and classification models. As users spend more time using the system, brain activity patterns may change. By continuously updating the model, the system can better adapt to these changes, improving its performance and stability.

[0050] Through the above technical solutions, this novel EEG signal decoding algorithm framework can effectively improve the information transmission rate of brain-computer interface spellers, reduce error output, and enhance brain-computer information transmission capabilities.

[0051] This invention, through the synergistic operation of the aforementioned technical features, enables effective judgment of EEG signal quality, avoids misclassification of low-quality signals, improves the information transmission rate and accuracy of the brain-computer interface speller, enhances brain-computer information transmission capabilities, and provides a more reliable algorithm framework for the application of brain-computer interface technology in multiple fields. It has significant innovation and practicality and should be protected by law.

[0052] The novel EEG signal decoding algorithm framework proposed in this invention is expected to bring significant technological advancements to the field of brain-computer interface spellers, greatly enhancing information transmission capabilities.

[0053] Firstly, regarding improving spelling speed, this framework effectively avoids misclassification caused by low-quality signals by pre-judging the quality of EEG signals. Traditional algorithms classify signals regardless of quality, requiring corrections like backspace after errors occur, severely impacting spelling efficiency. This framework intelligently selects high-quality signals for classification, reducing errors and enabling users to output more accurate characters per unit of time. For example, in practical applications, the process of repeatedly backspaced to correct frequent errors is simplified. Previously, only 10-15 characters could be output per minute; with the new framework, this number is expected to increase to 20-25 characters or even higher, significantly saving input time.

[0054] Secondly, from the perspective of enhancing brain-computer interface (BCI) information transmission capabilities, the signal quality assessment mechanism provides a more reliable data foundation for subsequent classification. High-quality EEG signals, after accurate classification, can more precisely reflect the user's intentions, reducing noise interference and misjudgments during information transmission. This means that the BCI spelling device is more accurate in understanding user commands, not only improving the accuracy of information transmission but also significantly enhancing the stability and reliability of the entire system. Users will experience a smoother and more natural interactive experience when using the BCI spelling device, much like smooth communication between people, propelling BCI technology from the laboratory to a wider range of practical applications, bringing more efficient and convenient solutions to fields such as assistive communication for people with disabilities and smart home control.

[0055] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0056] Furthermore, some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, mainframe computers, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0057] The electronic device includes: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the methods provided in any one or more of the above embodiments.

[0058] Figure 3 An exemplary structural diagram of the electronic device is disclosed. The electronic device includes one or more processors 1101, a memory 1102, an input device 1103, and an output device 1104. The various components are interconnected via a bus or other means (the diagram shows an example of bus connection). The processor 1101 can be used to execute instructions stored in the memory 1102 to control the overall operation of the electronic device. The memory 1102 may include a program storage area and a data storage area, wherein the program storage area stores the operating system and applications required for at least one function; the data storage area stores data created according to the use of the electronic device, etc. The memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as disk storage devices, flash memory devices, or other non-transitory solid-state storage devices. In some embodiments, the memory 1102 may also include storage resources located remotely to the processor and accessible via a network.

[0059] Input device 1103 can be used to receive input numerical or character information or user operation signals, such as a touch screen, keypad, mouse, trackpad, touchpad, indicator, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include display devices (such as liquid crystal displays, light-emitting diode displays, plasma displays, and optional touch screens), auxiliary lighting devices (such as LEDs), and haptic feedback devices (such as vibration motors), etc.

[0060] To facilitate user interaction, the electronic device may be configured to include a display device (such as an LCD or CRT monitor) and input devices such as a keyboard and pointing devices (e.g., a mouse or touchpad). Feedback can be any form of sensory feedback (e.g., visual feedback, auditory feedback); input may also be received via voice, touch, or other means.

[0061] This application also relates to a computer-readable medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the methods provided in any one or more of the above embodiments. This computer-readable medium may be a memory included in an electronic device, or it may be a standalone storage medium not assembled into the device.

[0062] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium, a computer-readable storage medium, or a combination of both. Examples include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. Specific examples of storage media may include, but are not limited to, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory, optical fibers, portable CD-ROMs, optical storage devices, magnetic storage devices, etc., or any suitable combination thereof.

[0063] Computer-readable media can store one or more programs that can be used by or in conjunction with an instruction execution system. The media can be permanent or non-permanent, removable or non-removable, and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data.

[0064] The computer program code used to implement the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Java, Smalltalk, and C++) and conventional procedural programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. The remote computer can be connected to the user's computer via any network (including a local area network or a wide area network) or can be connected to an external computer.

[0065] In the above embodiments, the functions can be implemented in whole or in part by software, hardware, firmware, or any combination thereof, for example, by using an application-specific integrated circuit, a general-purpose computer, or other similar hardware devices. In some embodiments, the software program of this application can be executed by a processor to implement the steps or functions; it can also be implemented by hardware, for example, as a circuit that works in conjunction with the processor to execute the steps or functions.

[0066] This application also provides a computer program product, including one or more computer programs / instructions, which, when executed by a processor, generate all or part of the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one storage medium to another via wired (e.g., DSL) or wireless (e.g., wireless, microwave) means. The computer-readable storage medium may be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive).

[0067] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0068] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device in software or hardware. Terms such as "first," "second," etc., are used only for distinguishing descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0069] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A high-efficiency decoding method for brain-computer interfaces based on signal quality pre-screening, characterized in that, The signal quality assessment module extracts time-domain, frequency-domain, and time-frequency-domain features from the acquired EEG signals, and uses support vector machine or other algorithms to construct a signal quality assessment model. Based on the set quality threshold, the quality of the EEG signal is judged to determine whether the signal should be sent to subsequent modules. Through the feature extraction and selection module, for EEG signals that have passed the quality assessment, independent components are further separated and features are extracted using methods such as independent component analysis; algorithms such as Relief-F are used to select the extracted features, calculate feature weights, and select a subset of features with higher weights to reduce feature dimensionality; The classification decision module uses algorithms such as Long Short-Term Memory Network to classify EEG signals after feature extraction and selection. During training, an adaptive learning rate adjustment strategy is used, combined with L1 and L2 regularization methods to prevent model overfitting, and outputs accurate EEG signal classification results. The adaptive adjustment module monitors the accuracy and stability of classification results in real time, based on user feedback; it also collects new EEG signal data regularly to update and optimize the signal quality assessment model and classification model.

2. An electronic device, characterized in that, The electronic device includes: One or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method as described in claim 1.

3. A computer-readable medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

4. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

Images