A multifunctional character input system based on hybrid brain-computer interface

Abstract

The present invention discloses a multifunctional character input system based on a hybrid brain-computer interface, including: a hybrid EEG signal stimulation module, which is used to stimulate the generation of a hybrid EEG signal of P300 and SSVEP, mainly for visual stimulation of Chinese and English characters and function keys; an EEG signal processing module, which mainly collects real-time EEG data of users and performs preprocessing, feature extraction, and signal classification on the EEG data; an asynchronous control module, which is used for system state switching, performs asynchronous control based on EOG, and can switch between the character input state and the idle state at any time; a Chinese and English character input module, which uses a hybrid EEG method of P300+SSVEP to input and select characters, uses Chinese Pinyin to spell Chinese characters, and can also spell English characters. The present invention uses a hybrid EEG-based method to input Chinese and English characters, which can effectively improve the input accuracy and efficiency of Chinese and English characters, and can also detect the user's state of consciousness.

Description

Technical Field

[0001] The present invention relates to the technical field of brain-computer interface, and in particular to a multifunctional character input system based on a hybrid brain-computer interface. Background Art

[0002] Brain-computer interface technology can convert EEG information into specific command information. When the brain's thinking and consciousness are stimulated by external visual stimuli, the human brain's nervous system can produce corresponding changes in EEG activity. By collecting EEG information and converting it into specific recognition commands after information processing, the human brain can interact with the outside world. The use of brain-computer interface technology can directly detect the user's EEG information, and characters can be input without the user's physical movements, so that communication with the outside world can be achieved.

[0003] At present, there are many problems with the character input system based on brain-computer interface. In the research of traditional brain-computer interface character input system, the focus is mainly on the input methods of English and numbers. The English character input function is single and can only be presented in the form of letters or English words. For users who are not familiar with English, this is undoubtedly an obstacle. Moreover, many previous character input systems use a single brain-computer interface technology, and the character recognition accuracy and recognition speed need to be improved, while the hybrid brain-computer interface can effectively improve these performance indicators. In the paradigm design of character input, it is mainly based on row and column matrices, which have a long stimulation time and low efficiency. In the synchronous character input brain-computer interface, the character input instructions and control instructions are set by the program, and the user needs to input characters according to the corresponding time interval. During the input task, the user will be fatigued, but cannot stop the input process, resulting in a decrease in input accuracy. It cannot determine the input state and idle state, and is not flexible enough in specific applications. For some critically ill patients in rehabilitation treatment, it is impossible to know their state of consciousness, which causes certain interference to rehabilitation treatment.

[0004] In response to the above problems, this paper invented a multifunctional character input system based on a hybrid brain-computer interface. It can provide a Chinese communication and interaction platform for these people, greatly facilitating users, their families and caregivers. It also promotes the rehabilitation treatment of users. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and propose a multifunctional character input system based on a hybrid brain-computer interface. Character recognition and input are performed by mixing SSVEP EEG signals with P300 EEG signals, and EOG is used for asynchronous state conversion, so that specific groups of people can use the system to perform asynchronous interaction with Chinese characters, and the consciousness state of specific groups of people can also be detected.

[0006] To achieve the above purpose, the technical solution provided by the present invention is: a multifunctional character input system based on a hybrid brain-computer interface, comprising:

[0007] A mixed EEG signal stimulation module is used to stimulate the generation of mixed EEG signals of P300 and SSVEP, and collect and save EEG signal data;

[0008] EEG signal processing module, used for data preprocessing, feature extraction and signal classification after EEG signal acquisition;

[0009] Asynchronous control module, used for switching between system input state and idle state, and performs asynchronous control based on EOG;

[0010] The Chinese and English character input module is used to input and display Chinese and English characters according to the classification and recognition results of the mixed EEG signal.

[0011] Furthermore, the mixed EEG signal stimulation module includes a SSVEP stimulation paradigm module and a P300 stimulation paradigm module, wherein:

[0012] The SSVEP stimulation paradigm module is used for primary area character selection. The SSVEP stimulation paradigm evenly distributes stimulation areas of the same size and style in the form of R rows and C columns, including English character stimulation areas, function key stimulation areas and digital stimulation areas, respectively. The row spacing and column spacing between these stimulation areas are kept the same, wherein 1<=R<=8, an integer is taken, and 1<=C<=8, an integer is taken; each English character stimulation area can contain N English characters, 1<=N<=8, an integer is taken, and the English characters are arranged in alphabetical order; in the function key stimulation area, it is displayed as shift text, and the stimulation area can be set to different function keys according to the system function; in the digital stimulation area, each stimulation area contains only one integer number, which is an integer from 0 to 9, and the digital stimulation areas are not adjacent; all stimulation areas flash at different fixed frequencies, and the frequency setting range is 8-15Hz, and the multiple frequencies between any frequencies cannot be the same. When the user looks at a stimulation area, an SSVEP electroencephalogram signal of the stimulation area being looked at will be generated, and the SSVEP electroencephalogram signal will be saved after it is generated;

[0013] The P300 stimulation paradigm module is used for secondary area character selection. The P300 stimulation area is uniformly distributed in the form of M rows and N columns with stimulation areas of the same size and style, where 1<=M<=8, an integer is taken, and 1<=N<=8, an integer is taken, and the row spacing and column spacing between each stimulation area are kept the same; when performing secondary area character selection, the user stares at a stimulation area, and after each stimulation area flashes once, a corresponding P300 electroencephalogram signal can be generated, and the electroencephalogram signal is saved.

[0014] Further, the EEG signal processing module includes a P300 EEG signal processing module and a SSVEP EEG signal processing module;

[0015] The processing process of the P300 EEG signal processing module and the SSVEP EEG signal processing module is the same, including the following steps:

[0016] 1) For EEG signal preprocessing, a transform domain adaptive filtering algorithm was used; after the P300 or SSVEP EEG signals were generated in the P300 and SSVEP stimulation paradigms, the collected EEG signals were filtered using the transform domain adaptive filtering algorithm; an N-order filter was designed, where N is the order. Define n as the moment; define X(n) as the input vector at moment n; define W(n) as the parameter matrix of the N-order transform domain adaptive filter at moment n; define y(n) as the filter output signal at moment n; define d(n) as the expected output signal at moment n; and define e(n) as the error signal at moment n.

[0017] X(n)=[x(n),x(n+1),…,x(n-N+1)] T (1)

[0018] W(n)=[w0(n),w1(n),…,w N-1 (n)] T (2)

[0019] Where x(j) is the corresponding input value at time n, j = n, n+1,…, n-N+1; w i (n) is the i-th weight at time n, i = 0, 1, ... N-1; T represents transposition; the output of the N-th order filter is:

[0020] y(n)=W(n) T X(n) (3)

[0021] The error signal of the filter is:

[0022] e(n)=d(n)-y(n)=d(n)-W(n) T X(n) (4)

[0023] According to the minimum mean square error criterion, the minimization objective function is defined as J(W), where W is the abbreviation of the parameter matrix W(n);

[0024] J(W)=E{|e(n)| 2}=E{|d(n)-W(n) T X(n)| 2} (5)

[0025] Taking the derivative of J(W) with respect to W and setting the derivative to 0, we get:

[0026]

[0027] in To find the derivative sign, the most suitable W can be obtained by derivation of formula (6); the EEG signal after transform domain adaptive filtering algorithm is used in EEG signal feature extraction;

[0028] 2) Fast Fourier transform is used to extract features from the EEG signals obtained by the transform domain adaptive filtering algorithm, and the data after feature extraction is classified using a support vector machine classifier;

[0029] 3) EEG signal classification, using support vector machine classifier as the classifier: constructing the classification decision hyperplane using the EEG signal features after fast Fourier transform, selecting the corresponding kernel function, and then selecting the classification decision function for classification;

[0030] Through the above-mentioned EEG signal processing, the SSVEP EEG signal and the P300 EEG signal can be identified, and the fixation area under the SSVEP stimulation paradigm and the P300 stimulation paradigm can be known; the corresponding English characters or numbers are output as the input of the Chinese and English character input module.

[0031] Further, the asynchronous control module is used to switch between the system character input state and the idle state. When the EOG signal is detected twice within 2 seconds, the system state is switched. The EOG signal detection needs to adopt waveform detection analysis, which includes the following steps:

[0032] 1) Extract the EOG signal, perform bandpass filtering, eliminate interference, and use the difference method to obtain the feature vector:

[0033] x′ t =x t -x t-1 (7)

[0034] Among them, x t ′ is the difference value of the tth sampling point, x t and x t-1 are the sampling values ​​of the t-th sampling point and the t-1-th sampling point respectively, and the corresponding linear feature vector F is obtained according to the extracted EOG signal;

[0035] 2) using a support vector machine classifier for classification processing, designing a support vector machine classifier based on the linear feature vector F, and classifying the feature vector F in the support vector machine classifier;

[0036] 3) Perform waveform detection on the linear feature vector F: There are obvious peak-valley characteristics in the waveform of the EOG signal after differential processing. When performing EOG signal waveform detection, first determine the position of the peak and trough of the EOG signal, calculate the interval time and accumulated energy between the peak and trough, and then calculate the peak and trough according to the pre-set minimum threshold D. min , maximum threshold D max and energy threshold E for waveform detection analysis;

[0037] 4) Decision control: After classification by the support vector machine classifier and waveform detection, the detection results show whether the user has detected the EOG signal continuously, that is, whether the EOG signal is detected twice within 2 seconds; if the EOG signal is detected twice within 2 seconds, the system state is switched, that is, switching between the idle state and the input state.

[0038] Furthermore, in the Chinese and English character input module, the Chinese-English spelling method is used to perform input spelling display;

[0039] When inputting Chinese characters, after the SSVEP stimulation paradigm and P300 stimulation paradigm are completed; if candidate Chinese character options are displayed in the display bar, each candidate Chinese character option is identified by a number, with an integer from 1 to 6; if the target Chinese character appears, the SSVEP stimulation paradigm is entered, and according to the number corresponding to the target Chinese character, the corresponding SSVEP electroencephalogram signal is generated by fixating on the SSVEP stimulation area of ​​the corresponding number, and the target Chinese character is output by processing the SSVEP electroencephalogram data; if the target Chinese character does not appear, the SSVEP stimulation paradigm and P300 stimulation paradigm are repeated until the target Chinese character appears; when inputting Chinese characters, the SSVEP stimulation paradigm is entered, and ... When inputting Chinese characters, by gazing at the function key shift area in the SSVEP stimulation paradigm, the Chinese character input state can be changed to the English character input state; after entering the Chinese and English character input module, the user can input English characters by gazing at the corresponding SSVEP stimulation paradigm and P300 stimulation paradigm; when stimulating in the SSVEP stimulation paradigm, if 2 EOG signals are detected within 2s, the system will switch the input state to the idle state, at which time the user can pause and rest the input; when 2 EOG signals are detected again within 2s, the system will switch the idle state to the input state, and Chinese and English characters can be input.

[0040] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0041] 1. The present invention fully considers the actual situation during use. The present invention adopts asynchronous brain-computer interface technology. When in use, the user can switch the character input state and the idle state according to the EOG signal, that is, by blinking twice in a row, providing a more flexible character interaction method.

[0042] 2. The present invention can also be used for consciousness status detection. For some patients who are in a coma or seriously injured but whose vision is not yet impaired, it provides a possibility for detecting the consciousness status of the patients.

[0043] 3. The present invention can be combined with other social software to enable users to interact with other people using Chinese characters, thereby improving the user's living conditions and enriching the entertainment methods.

[0044] 4. The present invention also has an auxiliary effect on the rehabilitation treatment of some users, can train the patient's state of consciousness, and is helpful for rehabilitation treatment.

[0045] In short, the present invention adopts a brain-computer interface technology based on a mixture of SSVEP EEG signals and P300 EEG signals, which has better accuracy and speed than a single character input system. In Chinese character input, input is based on Chinese Pinyin. This requires the use of 26 English letters, as well as common punctuation marks and some function keys; this will require the selection of multiple target characters in the stimulation paradigm, which has high requirements for stimulation time and greatly reduces the efficiency of Chinese character input. The two-level area paradigm adopted in the present invention greatly reduces the number of stimulation targets. In the first-level character selection, 12 SSVEP EEG signals of different frequencies are used to select the first-level area, and after entering the corresponding second-level area, P300 EEG signals are used for character selection. The use of this hybrid brain-computer interface paradigm improves the speed and accuracy of character selection and achieves a good application experience effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is the overall framework diagram of the system of the present invention.

[0047] Figure 2 Schematic diagram of the system of the present invention.

[0048] Figure 3 This is a diagram of the primary area stimulation paradigm of the system of the present invention.

[0049] Figure 4 A diagram of the secondary area stimulation paradigm of the system of the present invention.

[0050] Figure 5 Schematic diagram of the electrode cap for collecting EEG signals of the system of the present invention.

[0051] Figure 6This is a flow chart of EEG signal processing of the system of the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0053] like Figures 1 to 6 As shown, this embodiment discloses a multifunctional character input system based on a hybrid brain-computer interface, including the following functional modules:

[0054] A mixed EEG signal stimulation module is used to stimulate the generation of mixed EEG signals of P300 and SSVEP, and collect and save EEG signal data;

[0055] EEG signal processing module, used for data preprocessing, feature extraction, and signal classification after EEG signal acquisition;

[0056] Asynchronous control module, used for switching between system input state and idle state, and performs asynchronous control based on EOG;

[0057] The Chinese and English character input module is used to input and display Chinese and English characters according to the classification and recognition results of the mixed EEG signal.

[0058] In view of the above functional requirements, the following equipment was used: desktop computer / laptop computer, SynAmps2 amplifier, Quick-Cap32 electrode cap, NuAmps system and conductive glue, etc. The desktop computer / laptop computer is responsible for paradigm stimulation, data processing, character spelling and display, etc.; the EEG collector is responsible for EEG data collection.

[0059] During use, you must first wear the electrode cap and inject conductive glue into the 32 channels in the electrode cap to reduce impedance; then correctly connect the amplifier to the electrode cap, and correctly connect the desktop or laptop to the amplifier; the user looks at the stimulation area on the screen to input characters, and the input results will be fed back to the user to verify the correctness of the input.

[0060] The EEG collector used in this system consists of a SynAmps2 amplifier, a NuAmps system, and a Quick-Cap32 electrode cap.

[0061] The EEG signal acquisition electrodes used in this system are Fpz, Fz, Cz, Pz, Oz, Fp1, F3, C3, P3, O1, Fp2, F4, C4, P4, O2, F7, T3, T5, F8, T4, T6, A1, A2, etc., see Figure 5 shown.

[0062] Furthermore, the mixed EEG signal stimulation module includes a SSVEP stimulation paradigm module and a P300 stimulation paradigm module, wherein:

[0063] The SSVEP stimulation paradigm module is used for primary area character selection. The SSVEP stimulation paradigm evenly distributes stimulation areas of the same size and style in the form of R rows and C columns (where 1<=R<=8, an integer is taken; 1<=C<=8, an integer is taken), and the row spacing and column spacing between these stimulation areas are kept the same; in this embodiment, 12 stimulation areas of different sizes are evenly distributed in the form of 3 rows and 4 columns, including 5 English character stimulation areas, 1 function key stimulation area and 6 digital stimulation areas; each English character stimulation area can contain N English characters (1<=N<=8, an integer is taken), and the English characters are arranged in alphabetical order. In this embodiment, The example contains 6 English characters; the text "shift" is displayed in the function key stimulation area, and the stimulation area can be set to different function keys according to the system function; in the digital stimulation area, each stimulation area contains only one integer number (the number can be an integer from 0 to 9), and integers from 1 to 6 are used in this embodiment; the digital stimulation areas are not adjacent; the 12 areas flash at different fixed frequencies (the frequency setting range is 8-15Hz, and the multiple frequencies between any frequencies cannot be the same), when the user looks at a stimulation area, the SSVEP EEG signal of the stimulation area being looked at will be generated, and the SSVEP EEG signal will be saved after it is generated.

[0064] The P300 stimulation paradigm module is used for secondary area character selection. The P300 stimulation area is uniformly distributed in the form of M rows and N columns (where 1<=M<=8, an integer is taken; 1<=N<=8, an integer is taken) with stimulation areas of the same size and style. In this embodiment, 2 rows and 3 columns of distributed P300 stimulation areas are used, and the row spacing and column spacing between each stimulation area are kept the same. When selecting characters in the secondary area, the user stares at a stimulation area, and after each stimulation area flashes once, a corresponding P300 electroencephalogram signal can be generated, and the electroencephalogram signal is saved.

[0065] Further, the EEG signal processing module includes a P300 EEG signal processing module and a SSVEP EEG signal processing module;

[0066] The processing process of the P300 EEG signal processing module and the SSVEP EEG signal processing module is the same, including the following steps:

[0067] 1) For EEG signal preprocessing, a transform domain adaptive filtering algorithm was used; after the P300 or SSVEP EEG signals were generated in the P300 and SSVEP stimulation paradigms, the collected EEG signals were filtered using the transform domain adaptive filtering algorithm; an N-order filter was designed, where N is the order. Define n as the moment; define X(n) as the input vector at moment n; define W(n) as the parameter matrix of the N-order transform domain adaptive filter at moment n; define y(n) as the filter output signal at moment n; define d(n) as the expected output signal at moment n; and define e(n) as the error signal at moment n.

[0068] X(n)=[x(n),x(n+1),…,x(n-N+1)] T (8)

[0069] W(n)=[w0(n),w1(n),…,w N-1 (n)] T (9)

[0070] Where x(j) (j=n, n+1,…, n-N+1) is the corresponding input value at time n; w i (n)(i=0,1,…N-1) is the i-th weight at time n; T represents transposition. The output of the N-th order filter is:

[0071] y(n)=W(n) T X(n) (10)

[0072] The error signal of the filter is:

[0073] e(n)=d(n)-y(n)=d(n)-W(n) T X(n) (11)

[0074] According to the minimum mean square error criterion, the minimization objective function is defined as J(W), where W is the abbreviation of the parameter matrix W(n).

[0075] J(W)=E{|e(n)| 2}=E{|d(n)-W(n) T X(n)| 2} (12)

[0076] Taking the derivative of J(W) with respect to W and setting the derivative to 0, we get:

[0077]

[0078] in is the derivative symbol. The most suitable W can be obtained by the derivative of formula (13). The EEG signal after the transform domain adaptive filtering algorithm is used for EEG signal feature extraction;

[0079] 2) Fast Fourier transform is used to extract features from the EEG signals obtained by the transform domain adaptive filtering algorithm, and the data after feature extraction is classified using a support vector machine classifier;

[0080] 3) EEG signal classification, using support vector machine classifier as the classifier: constructing the classification decision hyperplane using the EEG signal features after fast Fourier transform, selecting the corresponding kernel function, and then selecting the classification decision function for classification;

[0081] Through the above-mentioned EEG signal processing, the SSVEP EEG signal and the P300 EEG signal can be identified, so that the fixation area under the SSVEP stimulation paradigm and the P300 stimulation paradigm can be known; the corresponding English characters or numbers are output as the input of the Chinese and English character input module;

[0082] The asynchronous control module is used to switch between the system character input state and the idle state. When the EOG signal is detected twice within 2 seconds, the system state is switched. The EOG signal detection requires waveform detection and analysis, which includes the following steps:

[0083] 1) Extract the EOG signal, perform bandpass filtering, eliminate interference, and use the difference method to obtain the feature vector:

[0084] x′ t =x t -x t-1 (14)

[0085] Among them, x′ t is the difference value of the tth sampling point, x t and x t-1 are the sampling values ​​of the t-th sampling point and the t-1-th sampling point respectively, and the corresponding linear feature vector F is obtained according to the extracted EOG signal;

[0086] 2) using a support vector machine (SVM) classifier for classification processing, designing a support vector machine (SVM) classifier based on the linear feature vector F; and classifying the feature vector F in the support vector machine (SVM) classifier;

[0087] 3) Perform waveform detection on the linear feature vector F: There are obvious peak-valley characteristics in the waveform of the EOG signal after differential processing. When performing EOG signal waveform detection, first determine the position of the peak and trough of the EOG signal, and calculate the interval time and accumulated energy between the peak and trough. According to the pre-set minimum threshold D min , maximum threshold D max and energy threshold E for waveform detection analysis;

[0088] 4) Decision control: After classification and waveform detection by the support vector machine (SVM) classifier, the detection results show whether the user has detected the EOG signal continuously (i.e., the EOG signal is detected twice within 2 seconds); if the EOG signal is detected twice within 2 seconds, the system state can be switched, i.e., switching between the idle state and the input state;

[0089] Furthermore, in the Chinese and English character input module, the Chinese-English spelling method is used to perform input spelling display.

[0090] When inputting Chinese characters, after the SSVEP stimulation paradigm and P300 stimulation paradigm are completed; if candidate Chinese character options are displayed in the display bar, each candidate Chinese character option is identified by a number (identified by an integer from 1 to 6); if the target Chinese character appears, the SSVEP stimulation paradigm is entered, and according to the number corresponding to the target Chinese character, the corresponding SSVEP electroencephalogram signal is generated by fixating on the SSVEP stimulation area of ​​the corresponding number, and the target Chinese character is output by processing the SSVEP electroencephalogram data; if the target Chinese character does not appear, the SSVEP stimulation paradigm and P300 stimulation paradigm can be repeated until the target Chinese character appears; when inputting English characters, the Chinese character input state can be changed to the English character input state by fixating on the function key shift area in the SSVEP stimulation paradigm; after entering the English character input module, the user can input English characters by fixating on the corresponding SSVEP stimulation paradigm and P300 stimulation paradigm. When stimulating in the SSVEP stimulation paradigm, if two EOG signals are detected within 2 seconds, the system will switch the input state to the idle state, and the user can pause the input and take a rest; when two EOG signals are detected again within 2 seconds, the system will switch the idle state to the input state, and Chinese and English characters can be input.

[0091] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A multifunctional character input system based on a hybrid brain-computer interface, characterized in that: include: A mixed EEG signal stimulation module is used to stimulate the generation of mixed EEG signals of P300 and SSVEP, and collect and save EEG signal data; the mixed EEG signal stimulation module includes an SSVEP stimulation paradigm module and a P300 stimulation paradigm module, wherein: The SSVEP stimulation paradigm module is used for primary area character selection. The SSVEP stimulation paradigm evenly distributes stimulation areas of the same size and style in the form of R rows and C columns, including English character stimulation areas, function key stimulation areas and digital stimulation areas, respectively. The row spacing and column spacing between these stimulation areas are kept the same, wherein 1<=R<=8, an integer is taken, and 1<=C<=8, an integer is taken; each English character stimulation area can contain N English characters, 1<=N<=8, an integer is taken, and the English characters are arranged in alphabetical order; in the function key stimulation area, it is displayed as shift text, and the stimulation area can be set to different function keys according to the system function; in the digital stimulation area, each stimulation area contains only one integer number, which is an integer from 0 to 9, and the digital stimulation areas are not adjacent; all stimulation areas flash at different fixed frequencies, and the frequency setting range is 8-15Hz, and the multiple frequencies between any frequencies cannot be the same. When the user looks at a stimulation area, an SSVEP electroencephalogram signal of the stimulation area being looked at will be generated, and the SSVEP electroencephalogram signal will be saved after it is generated; The P300 stimulation paradigm module is used for secondary area character selection. The P300 stimulation area is uniformly distributed in the form of M rows and N columns with stimulation areas of the same size and style, where 1<=M<=8, an integer is taken, and 1<=N<=8, an integer is taken, and the row spacing and column spacing between each stimulation area are kept the same; when performing secondary area character selection, the user stares at a certain stimulation area, and after each stimulation area flashes once, a corresponding P300 electroencephalogram signal can be generated, and the electroencephalogram signal is saved; An EEG signal processing module is used for data preprocessing, feature extraction and signal classification after EEG signal collection; the EEG signal processing module includes a P300 EEG signal processing module and a SSVEP EEG signal processing module; The processing process of the P300 EEG signal processing module and the SSVEP EEG signal processing module is the same, including the following steps: 1) Preprocessing of EEG signals, using transform domain adaptive filtering algorithm; after generating P300 or SSVEP EEG signals in P300 and SSVEP stimulation paradigms, the collected EEG signals are filtered using transform domain adaptive filtering algorithm; design an N-order filter; define n as time; define X(n) as the input vector at time n; define W(n) as the parameter matrix of N-order transform domain adaptive filtering at time n; define y(n) as the filter output signal at time n; define d(n) as the expected output signal at time n; define e(n) as the error signal at time n; X(n)=[x(n),x(n+1),…,x(n-N+1)] T (1) W(n)=[w0(n),w1(n),…,w N-1 (n)] T (2) Where x(j) is the corresponding input value at time n, j = n, n+1,…, n-N+1; w i (n) is the i-th weight at time n, i = 0, 1, ... N-1; T represents transposition; the output of the N-th order filter is: y(n)=W(n) T X(n)(3) The error signal of the filter is: e(n)=d(n)-y(n)=d(n)-W(n) T X(n)(4) According to the minimum mean square error criterion, the minimization objective function is defined as J(W), where W is the abbreviation of the parameter matrix W(n); J(W)=E{|e(n)| 2 }=E{|d(n)-W(n) T X(n)| 2 }(5) Taking the derivative of J(W) with respect to W and setting the derivative to 0, we get: in To find the derivative sign, the most suitable W can be obtained by derivation of formula (6); the EEG signal after transform domain adaptive filtering algorithm is used in EEG signal feature extraction; 2) Fast Fourier transform is used to extract features from the EEG signals obtained by the transform domain adaptive filtering algorithm, and the data after feature extraction is classified using a support vector machine classifier; 3) EEG signal classification, using support vector machine classifier as the classifier: constructing the classification decision hyperplane using the EEG signal features after fast Fourier transform, selecting the corresponding kernel function, and then selecting the classification decision function for classification; Through the above-mentioned EEG signal processing, the SSVEP EEG signal and the P300 EEG signal can be identified, and the fixation area under the SSVEP stimulation paradigm and the P300 stimulation paradigm can be known; the corresponding English characters or numbers are output as the input of the Chinese and English character input module; Asynchronous control module, used for switching between system input state and idle state, and performs asynchronous control based on EOG; The Chinese and English character input module is used to input and display Chinese and English characters according to the classification and recognition results of the mixed EEG signal.

2. A multifunctional character input system based on a hybrid brain-computer interface according to claim 1, characterized in that: The asynchronous control module is used to switch between the system character input state and the idle state. When the EOG signal is detected twice within 2 seconds, the system state is switched. The EOG signal detection needs to adopt waveform detection analysis, which includes the following steps: 1) Extract the EOG signal, perform bandpass filtering, eliminate interference, and use the difference method to obtain the feature vector: x t '=x t -x t-1 (7) Among them, x t ' is the difference value of the tth sampling point, x t and x t-1 are the sampling values ​​of the t-th sampling point and the t-1-th sampling point respectively, and the corresponding linear feature vector F is obtained according to the extracted EOG signal; 2) using a support vector machine classifier for classification processing, designing a support vector machine classifier based on the linear feature vector F, and classifying the feature vector F in the support vector machine classifier; 3) Perform waveform detection on the linear feature vector F: There are obvious peak-valley characteristics in the waveform of the EOG signal after differential processing. When performing EOG signal waveform detection, first determine the position of the peak and trough of the EOG signal, calculate the interval time and accumulated energy between the peak and trough, and then calculate the peak and trough according to the pre-set minimum threshold D. min , maximum threshold D max and energy threshold E for waveform detection analysis; 4) Decision control: After classification by the support vector machine classifier and waveform detection, the detection results show whether the user has detected the EOG signal continuously, that is, whether the EOG signal is detected twice within 2 seconds; if the EOG signal is detected twice within 2 seconds, the system state is switched, that is, switching between the idle state and the input state.

3. A multifunctional character input system based on hybrid brain-computer interface according to claim 1, characterized in that: In the Chinese and English character input module, the Chinese Pinyin spelling method is used to input spelling display; When inputting Chinese characters, after the SSVEP stimulation paradigm and P300 stimulation paradigm are completed; if candidate Chinese character options are displayed in the display bar, each candidate Chinese character option is identified by a number, and an integer from 1 to 6 is identified; if the target Chinese character appears, the SSVEP stimulation paradigm is entered, and according to the number corresponding to the target Chinese character, the SSVEP stimulation area corresponding to the number is fixed to generate the corresponding SSVEP EEG signal, and the target Chinese character is output by processing the SSVEP EEG data; If the target Chinese character did not appear, the SSVEP stimulation paradigm and P300 stimulation paradigm were repeated until the target Chinese character appeared; When inputting English characters, gazing at the function key shift area in the SSVEP stimulation paradigm can change the Chinese character input state to the English character input state; after entering the Chinese and English character input module, the user can input English characters by gazing at the corresponding SSVEP stimulation paradigm and P300 stimulation paradigm; when stimulating in the SSVEP stimulation paradigm, if two EOG signals are detected within 2s, the system will switch the input state to the idle state, at which time the user can pause and rest the input; When the re-detection generates two EOG signals within 2 seconds, the system switches the idle state to the input state, and Chinese and English characters can be input.

Images