User-friendly brain-computer interface system

Abstract

The invention discloses a user-friendly brain-computer interface system, and relates to the technical field of brain-computer interfaces. The system comprises a stimulation presentation module, a signal acquisition module, a signal processing module and a signal decoding module. The stimulus presentation module is used for presenting static color block stimulus at different positions on the display equipment; the signal acquisition module is used for acquiring scalp surface electroencephalogram signals when a user watches a color block; the signal processing module adopts a TRCA algorithm to carry out signal-to-noise ratio enhancement on the electroencephalogram signals, and an electrode channel weight coefficient is obtained by solving a target function; the signal decoding module adopts a Manhattan distance method to describe electroencephalogram signal characteristic differences stimulated by different color blocks. Visual fatigue is remarkably reduced, decoding performance is stable, color combination is visual, and requirements for different control instruction numbers can be flexibly met; static color block stimulation does not need to occupy a large number of computing resources, the high requirement of traditional flicker stimulation for computer performance is met, and the method is suitable for more hardware environments.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of brain-computer interface, and particularly relates to a user-friendly brain-computer interface system. BACKGROUND

[0002] A brain-computer interface (BCI) decodes the brain intention by collecting the neural activity of the brain (epidermis or internal brain), and controls external devices according to the decoding result. At present, according to whether the electrode needs to be implanted into the brain, the collection method of brain neural activity signal is divided into two kinds of invasive collection and non-invasive collection. Compared with the invasive scheme which needs to be operated, the non-invasive scheme represented by electroencephalogram (EEG) only needs to record the potential signal on the scalp surface, and does not need to perform high-risk surgery. However, since the non-invasive scheme cannot collect the complex activity in the deep brain, the encoding control instruction becomes particularly important.

[0003] At present, the methods for encoding control instructions mainly include the following: motor imagery (MI), steady-state visual evoked potential (SSVEP), coded visual evoked potential (CVEP), and improved schemes based on these schemes (steady-state peripheral visual evoked potential, high-frequency steady-state visual evoked potential, etc.). The BCI system based on MI realizes the output of different control instructions by imagining the movement of the limbs. Although this method avoids external pattern stimulation, a large amount of training data is needed to achieve high decoding accuracy. In addition to the BCI system based on MI, most BCI systems need to encode the control instruction into the corresponding movement mode to obtain distinguishable electroencephalogram signals. These BCI schemes using movement mode stimulation greatly reduce the collection time of training data, but the pattern stimulation in the form of flickering, displacement and the like will bring strong visual fatigue to the user. Reducing the visual fatigue caused by the BCI based on VEP is the most important goal of optimizing and improving this kind of BCI system.

[0004] Using high-frequency stimulation--steady-state motion visual evoked potential (SSMVEP) can alleviate the visual fatigue caused by the visual evoked potential (VEP) based biometric recognition (BCIs). However, these stimulation paradigms that alleviate visual fatigue still cannot get rid of the dependence on movement patterns, which still causes a certain degree of visual fatigue to the user. In addition, the basic characteristics of the computer screen also make it difficult to stably present the stimulation represented by flickering, which occupies a large amount of computing resources.

[0005] The exploration of static pattern stimulation paradigm in the field of BCI is still in the early stage. According to the current development process, the best design scheme of VEP-based BCI is to explore the combination of static stimulation, which can rely on image movement and induce distinguishable brain electrical signals. The research of Ruxue Li on the visual illusion paradigm shows that the paradigm of inducing motion illusion can induce separable event-related potential (ERP) signals. These visual illusion paradigm studies prove that the brain electrical signals induced by some specific static patterns have classifiable signal characteristics, therefore, the present scheme proposes a user-friendly brain-computer interface system. SUMMARY

[0006] The present application provides a user-friendly brain-computer interface system, and designs a more comfortable SSVEP stimulation paradigm. The stimulation paradigm sets different color regions on the target coded by the same flicker frequency and phase at different positions, so as to realize the decoding of electroencephalogram signals in this scenario. The mcVEP paradigm is used as the main research paradigm, and the characteristics of EEG signals in time domain and frequency domain under two color combination paradigms are compared. The ETRCA method is used to realize the detection of EEG signals under the mcVEP paradigm, and the disadvantage of VEP-BCI that brings visual discomfort to users is solved.

[0007] To solve the above technical problems, the present application is realized by the following technical scheme:

[0008] The user-friendly brain-computer interface system of the present application takes the mcVEP paradigm as the core, combines the TRCA algorithm and the Manhattan distance method, and constructs a complete brain-computer interface system. The specific technical scheme is as follows:

[0009] 1. System composition

[0010] The present system includes a stimulation presentation module, a signal acquisition module, a signal processing module and a signal decoding module, and the functions and designs of each module are as follows:

[0011] Stimulation presentation module

[0012] Hardware carrier: 27-inch LCD with a resolution of 1920x1080 and a refresh rate of 60Hz; the difference in color pattern display of different brands of display has little effect on decoding performance, so there is no need to limit specific brands.

[0013] Stimulus design: Static color blocks are presented at different locations on the display. The size of the color blocks is uniformly 200 pixels × 200 pixels. All color blocks use the same flashing frequency and phase encoding. Different control commands are distinguished only by color combinations. The selected color combinations are "black and white combination" or "red and white combination" (red RGB value is (255,0,0), white RGB value is (255,255,255)).

[0014] Program Development: The stimulus presentation program was developed using the Psychphysics Toolbox Version 3 tool in MATLAB software. The position and number of color blocks (corresponding to the number of control commands) can be flexibly adjusted to adapt to different use cases.

[0015] Signal acquisition module

[0016] Equipment selection: No specific equipment restrictions, compatible with mainstream EEG acquisition equipment on the market; this embodiment uses Neuroscan SynAmps2 64-256 EEG amplifier, which supports a single device connection of 64-electrode EEG cap, or 4 devices connected in parallel to achieve 256-electrode acquisition.

[0017] Acquisition method: The user wears an EEG cap to directly collect EEG signals from the scalp surface; only electrode channels corresponding to visual-related brain regions (such as the occipital lobe and parietal lobe visual cortex regions, see Figure 2 for specific channel distribution) are selected to reduce redundant signal interference.

[0018] Signal processing module

[0019] Core Algorithm: The TRCA algorithm is used to enhance the signal-to-noise ratio of EEG signals. The TRCA algorithm suppresses linear noise and reduces the amplitude error between channels by linear superposition of multiple channels. It can effectively enhance the signal with only a small amount of training data (under the traditional SSVEP paradigm, this algorithm has achieved a signal transmission rate (ITR) of over 325 bits / min).

[0020] Algorithm implementation: The superposition coefficient of electrode channels in each brain region is obtained by solving the following formula:

[0021] Objective function:

[0022] In formula (1) This refers to the EEG signal (dimension: number of channels × time point) from a single experiment in the first phase (data training phase). Weighting coefficients. It can be obtained using formula (2). Number of trials. (2)

[0023] represents the signal similarity between the i-th and j-th trials and different channels. The eigenvector corresponding to the largest eigenvalue of is the weight coefficient of each channel in the brain region .

[0024] Number decoding (the use of TRCA algorithm, not limited to the use of this algorithm)

[0025] In the first phase, the training of spatial filters under each frequency and the construction of reference templates are completed . After that, signal decoding is the key to frequency determination for unknown EEG segments in the test phase. First, for each test trial to be classified , the optimal spatial filter corresponding to each frequency f is used for projection, respectively, to obtain a one-dimensional filtered time-domain signal

[0026] Then, the projection signal is calculated with the reference template under the frequency Pearson correlation coefficient:

[0027] Here and represent covariance and variance operations, respectively. Finally, the correlation coefficients under all candidate frequencies are compared, and the frequency with the largest correlation coefficient is selected as the classification result of the trial, that is

[0028] Since the TRCA projected signal can maximize the consistency between the same frequency trials, the correlation coefficient usually presents a significant peak at the true stimulation frequency, thereby ensuring the accuracy of classification.

[0029] The present application has the following beneficial effects compared with the prior art:

[0030] (1) Significantly reduce visual fatigue: static color block stimulation is adopted to avoid flickering, displacement and other dynamic patterns of traditional VEP paradigm, completely get rid of the dependence on motion mode, and greatly improve the comfort of long-term use of users;

[0031] (2) Stable decoding performance: through the design of "same frequency phase + different color combination", combined with the signal-to-noise ratio enhancement ability of TRCA algorithm, even if the sampling time exceeds 0.6 seconds, the decoding accuracy can still remain stable, without the need to prolong the sampling time to improve the performance, reducing the waiting time of users;

[0032] ​(3) high ease of use: the TRCA algorithm only needs a small amount of training data, and users can use it without a large amount of learning; color combinations are intuitive and can be flexibly adapted to different control instruction quantity requirements;

[0033] (4) reduce system burden: static color block stimulation does not need to occupy a large amount of computing resources, solving the high requirement of traditional flicker stimulation on computer performance and adapting to more hardware environments.

[0034] Of course, any product implementing the present application does not necessarily need to achieve all the advantages described above at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed for the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0037] Figure 1 User interface effect diagram and design parameter schematic diagram of the stimulation paradigm;

[0038] Figure 2 Electrode channel selection schematic diagram for signal acquisition;

[0039] Figure 3 Experimental process schematic diagram;

[0040] Figure 4 Decoding accuracy change curve of different sampling times under black and white color combinations;

[0041] Figure 5 Decoding accuracy change curve of different sampling times under red and white color combinations. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0044] In view of the disadvantage that VEP-BCI brings visual discomfort to users, the multi-color visual evoked potential (mcVEP) paradigm is adopted as the main research paradigm in the present research. The characteristics of EEG signals in the time domain and frequency domain are compared under two color combination paradigms. In addition, the detection of EEG signals under the mcVEP paradigm is realized through the ETRCA method.

[0045] Stimulus paradigm:

[0046] The present application increases the amplitude feature difference of EEG signals corresponding to different targets by setting different color blocks (such as Figure 1 shown) at different positions. In order to compare the performance of brain-computer interface (BCI) systems under different color combinations, two color combination encoding control instructions (black and white combination, red and white combination) are adopted in the present research. The visual stimulus is presented through a 27-inch liquid crystal display (LCD) (resolution: 1920x1080, refresh rate: 60 Hz). The size of each color pattern is 200 pixels x 200 pixels. It should be noted that there will be some differences when different brands of displays display the same color pattern, but this factor has little effect on the decoding performance of EEG signals under the SSPVEP paradigm. The stimulus program is developed using the Psychphysics Toolbox Version 3 in MATLAB. Figure 1 The effect diagram of the user interface and some design parameters of the proposed paradigm are shown.

[0047] Detection algorithm:

[0048] With the continuous exploration of biological signal enhancement algorithms, the signal enhancement algorithm represented by the event-related analysis (TRCA) algorithm can effectively enhance the signal through a small amount of training, and the signal can be classified with high precision through simple correlation analysis at the detection end. With the help of this algorithm, the transmission rate (ITR) of the signal under the traditional full-target full-flashing SSVEP stimulus paradigm has exceeded 325 bits / min. The TRCA algorithm focuses more on linear processing of signals through linear superposition between multiple channels, suppresses linear noise in the signal, and reduces the amplitude error between channels. The EEG signals collected under the paradigm proposed in the present application can be further enhanced using the TRCA algorithm to improve the signal-to-noise ratio of EEG. Since the amplitude of the SSVEP signal characteristics generated by the brain under the stimulation of different color rectangles in the present paradigm has certain differences, the Manhattan distance method is used to describe the characteristics of EEG signals corresponding to different stimuli at the decoding end.

[0049] This study compared two color combination paradigms and found that extending the sampling time did not yield more distinctive EEG signals. To further verify this conclusion, this study also used the TRCA algorithm to demonstrate the changes in the detection accuracy of the proposed paradigm for EEG signals under different sampling times. Figure 4 (Black & White B&W) Figure 5 (Red-white R&W) As can be seen, when the sampling time exceeds 0.6 seconds, the accuracy of EEG signal detection under the two proposed paradigms (black-and-white B&W, red-and-white R&W) begins to gradually decrease. Compared with the VEP paradigm of dynamic stimulation, increasing the sampling time under the mcVEP paradigm does not improve the accuracy of EEG signal detection.

[0050] The specific implementation of the user-friendly brain-computer interface proposed in this invention is described from the following aspects: stimulation paradigm, experimental procedure, signal acquisition, and signal processing.

[0051] Stimulus Paradigm:

[0052] by Figure 1 Taking the brain-computer interface paradigm as an example, two stimulus targets are presented on the display (the actual number could be different in a real system). These two different stimulus targets can each correspond to two control commands. The specific distribution method is not limited in this invention. In a real interactive system, the position, shape, color, and corresponding control command type of the two stimulus targets can be adjusted according to the specific user scenario. Therefore, all characters here are replaced by numbers such as 1 and 2 to represent the corresponding control commands.

[0053] Experimental procedure:

[0054] like Figure 3 When a user uses the brain-computer interface system proposed in this invention, the output of each control command is divided into two stages. The first stage is called the training stage, in which the user first needs to gaze at each target for a few seconds (in this invention, the number of targets is 5), and then repeat gazing at all targets about 10 times (e.g., ...). Figure 3At the start of the experiment, a crosshair indicator appears on the screen for two seconds, prompting the user to focus their gaze on the left crosshair. After two seconds, two colored blocks appear on the screen: red RGB(255,0,0) and white (255,255,255). These colored blocks last for five seconds, during which the user must focus their gaze on either the left or right side based on the previously displayed crosshair indicator. One experiment lasts a total of 2 seconds + 5 seconds (considered one block), and a total of 10 such experiments are required. During this phase, the computer calibrates the user's EEG data, completing the algorithm training process. The second phase is the user's usage phase. In this phase, the user does not need to retrain the data. The user can choose to focus on the target corresponding to the desired control command or character, and the computer will output the corresponding control command or character. The duration of the two phases for each target output may vary depending on the specific system algorithm and the number of control commands implemented.

[0055] Signal Acquisition

[0056] This invention does not impose specific limitations on the devices used for acquiring EEG signals. Most commercially available EEG signal acquisition devices can meet the paradigm proposed in this invention. For the purpose of illustrating the embodiments, the Neuroscan SynAmps2 64-256-channel EEG amplifier is used here. This device supports a maximum of 64 electrodes in the EEG cap and allows four devices to be connected in parallel to acquire EEG signals from 256 electrodes. During use, the user needs to wear a acquisition headgear (EEG cap). The EEG cap directly acquires EEG signals from the scalp surface and sends them to the signal amplifier for processing. Finally, the processed signal is sent to the computer. Under the paradigm proposed in this invention, only electrode channels corresponding to visually related brain regions need to be selected (e.g., ...). Figure 2 (As shown).

[0057] Signal processing (use of the TRCA algorithm, but not limited to using this algorithm)

[0058] The TRCA algorithm obtains the superposition coefficient of electrode channels in each brain region by solving formula (1). (1)

[0059] In formula (1) This refers to the EEG signal (dimension: number of channels × time point) from a single experiment in the first phase (data training phase). Weighting coefficients. It can be obtained using formula (2). Number of trials. (2)

[0060] The signal similarity between different channels between the i-th and j-th trials is represented. Finally, The eigenvector corresponding to the largest eigenvalue of is the weight coefficient of each channel in the brain region .

[0061] Signal decoding (use of the TRCA algorithm, not limited to using this algorithm)

[0062] In the first stage, the spatial filter of each frequency is trained and the reference template is constructed After the training of the spatial filter of each frequency and the construction of the reference template, signal decoding is the key to frequency determination of unknown EEG segments in the test stage. First, for each test trial to be classified , the optimal spatial filter corresponding to each frequency f is used for projection to obtain a one-dimensional filtered time-domain signal

[0063] Then, the projection signal is calculated with the reference template of the frequency Pearson correlation coefficient:

[0064] Here and represent covariance and variance operations, respectively. Finally, the correlation coefficients of all candidate frequencies are compared, and the frequency with the largest correlation coefficient is selected as the classification result of the trial, that is

[0065] Since the TRCA projected signal can maximize the consistency between the same frequency trials, the correlation coefficient usually presents a significant peak at the true stimulation frequency, thereby ensuring the accuracy of the classification.

[0066] The specific implementation process of the present application will be described in detail below in combination with the drawings and experimental data:

[0067] (I) Stimulation paradigm implementation

[0068] Control instruction mapping: set 2 stimulation targets (scalable to a larger number), respectively corresponding to 2 control instructions (such as "left" and "right"), and identified by serial numbers "1" and "2";

[0069] ​Stimulus parameters: Color patch size 200 pixels x 200 pixels, black and white combination (black RGB (0,0,0), white RGB (255,255,255)) or red and white combination (red RGB (255,0,0), white RGB (255,255,255)), flicker frequency 15 Hz, phase 0°;

[0070] Presentation logic: 2 color patches were presented synchronously on a 27-inch LCD display by MATLAB Psychphysics Toolbox V3 program, with positions at left (x=300 pixels, y=540 pixels) and right (x=1620 pixels, y=540 pixels) of the screen, respectively.

[0071] (II) Experimental procedure implementation

[0072] The experiment was divided into training phase and use phase, the specific steps are as follows:

[0073] Training phase (calibration phase)

[0074] Preparation: The user wears Neuroscan SynAmps2 EEG cap, adjusts the electrode contact degree, and ensures the stability of signal acquisition;

[0075] Single block: The screen first displays a cross cursor for 2 seconds (prompting the gaze direction, such as the left cross corresponding to target 1 and the right cross corresponding to target 2), and then displays a color patch for 5 seconds, and the user gazes at the corresponding color patch according to the cross prompt;

[0076] Repeated training: Complete 10 blocks (including gaze training of all stimulus targets), and the computer completes EEG data calibration through TRCA algorithm to generate reference templates for each target.

[0077] Use phase (instruction output phase)

[0078] The user does not need to retrain, and directly gazes at the color patch corresponding to the output instruction;

[0079] The signal acquisition module acquires EEG signals in real time, which are enhanced by the signal processing module (TRCA algorithm), and the relevant coefficients are calculated by the signal decoding module to output the corresponding control instructions, and the whole process response time is ≤0.6 seconds.

[0080] (III) Signal acquisition and processing implementation

[0081] Signal acquisition: Neuroscan SynAmps2 64 lead EEG amplifier is adopted, the sampling rate is 500Hz, the filtering range is 0.1-100Hz; 16 visual related electrodes (O1, O2, Oz, P3, P4, Pz, PO3, PO4, POz, PO7, PO8, CB1, CB2, mid-occipital, mid-parietal, inferior temporal) are selected, and the reference electrode is placed in the left ear mastoid;

[0082] Signal pre-processing: the collected EEG signal is subjected to 50Hz power frequency filtering, and the artifacts (such as electromyogram and electrooculogram) are removed (independent component analysis (ICA) method is adopted);

[0083] TRCA algorithm execution: the calculation of formula (1)-(2) is realized through MATLAB, the electrode channel weight coefficient is obtained, and the signal enhancement is completed;

[0084] Decoding verification: for black and white and red and white two color combinations, 10 healthy subjects (20-30 years old, normal vision) are tested respectively, each subject completes 50 times of use stage test, and the results show that the average decoding accuracy of the two color combinations is all greater than or equal to 85% (when the sampling time is 0.6 seconds), and the visual fatigue score (visual analog scale VAS) is reduced by more than 60% compared with the traditional SSVEP mode.

[0085] The preferred embodiments disclosed above are only used to help explain the present application. The preferred embodiments do not describe all the details, and the application is not limited to the specific embodiments described. Obviously, according to the content of the specification, many modifications and changes can be made. The specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and utilize the present application. The present application is limited only by the claims and their entire scope and equivalents.

Claims

1. A user-friendly brain-computer interface system, characterized in that, It includes a stimulus presentation module, a signal acquisition module, a signal processing module, and a signal decoding module; The stimulus presentation module is used to present static color block stimuli at different positions on the display device. All color blocks use the same flashing frequency and phase encoding, and the color combination is selected from black and white combination or red and white combination. The signal acquisition module is used to acquire the electroencephalogram (EEG) signals on the scalp surface when the user is gazing at the color block, and only selects the electrode channels corresponding to the visually related brain regions. The signal processing module uses the TRCA algorithm to enhance the signal-to-noise ratio of the EEG signal and obtains the electrode channel weight coefficients by solving the objective function. The signal decoding module uses the Manhattan distance method to describe the differences in EEG signal characteristics of different color block stimuli, and combines the Pearson correlation coefficient to output control commands.

2. The user-friendly brain-computer interface system according to claim 1, characterized in that, The display device of the stimulation presentation module is a 27-inch LCD monitor with a resolution of 1920×1080 and a refresh rate of 60Hz; the color block size is 200 pixels × 200 pixels, the RGB value of red is (255,0,0), the RGB value of white is (255,255,255), and the RGB value of black is (0,0,0).

3. The user-friendly brain-computer interface system according to claim 1, characterized in that, The signal acquisition module uses a Neuroscan SynAmps2 64-256 EEG amplifier, which is used in conjunction with an EEG cap to acquire EEG signals. The sampling rate is 500Hz and the filtering range is 0.1-100Hz.

4. The user-friendly brain-computer interface system according to claim 1, characterized in that, The TRCA algorithm of the signal processing module solves for the weight coefficients using the following formula: (1) In formula (1) This refers to the EEG signal from a single experiment during the first stage, i.e., the data training stage. Its dimensions are: number of channels × time point; weighting coefficients. It can be obtained using formula (2); Number of training trials; For the signal similarity between the channels in the i-th and j-th trials; finally, The eigenvector corresponding to the largest eigenvalue is the weight coefficient of each brain region channel. .

5. The user-friendly brain-computer interface system according to claim 1, characterized in that, The decoding process of the signal decoding module includes: projecting the EEG signal from the test phase through a spatial filter, calculating the Pearson correlation coefficient with the reference template constructed in the training phase, and selecting the color block corresponding to the frequency with the largest correlation coefficient as the output result.

6. The user-friendly brain-computer interface system according to claim 1, characterized in that, It also includes an experimental process control module, which is divided into a training phase and a usage phase: the training phase completes the calibration by performing 10 "2-second crosshair indication + 5-second color block gaze" blocks; the usage phase directly outputs the control command corresponding to the user's gaze on the color block, with a response time ≤0.6 seconds.

Images